[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:37:21 dist-test-slave-h6n0 krb5kdc[21736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:37:21 dist-test-slave-h6n0 krb5kdc[21736](info): set up 2 sockets
Aug 10 14:37:21 dist-test-slave-h6n0 krb5kdc[21736](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 14:37:22.528194 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:37:22 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:37:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:37:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:37:22.746896 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38057
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:38057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:23.090730 21755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:23.091012 21755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:23.091106 21755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:23.098224 21755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:37:23.098335 21755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:23.098381 21755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:23.098441 21755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:37:23.098486 21755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:37:23.108220 21755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:38057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:23.109427 21755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:23.110767 21755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:23.122732 21770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:23.122781 21767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:23.122679 21775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:23.128180 21755 server_base.cc:1034] running on GCE node
I20260810 14:37:23.129101 21755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:23.130702 21755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:23.131968 21755 hybrid_clock.cc:648] HybridClock initialized: now 1786372643131920 us; error 113 us; skew 500 ppm
I20260810 14:37:23.132378 21755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:23 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372643, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:23.136370 21755 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:37:23.137912 21755 webserver.cc:460] Webserver started at http://127.21.55.126:33747/ using document root <none> and password file <none>
I20260810 14:37:23.138701 21755 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:23.138819 21755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:23.139087 21755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:23.142720 21755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "2fdae4323510473eb4a2fbeca78516bd"
format_stamp: "Formatted at 2026-08-10 14:37:23 on dist-test-slave-h6n0"
server_key: "17d29cb82ae71b04677c53f4312c5083"
server_key_iv: "dbbb140569b4255cebb9669562588119"
server_key_version: "encryptionkey@0"
I20260810 14:37:23.143667 21755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "2fdae4323510473eb4a2fbeca78516bd"
format_stamp: "Formatted at 2026-08-10 14:37:23 on dist-test-slave-h6n0"
server_key: "17d29cb82ae71b04677c53f4312c5083"
server_key_iv: "dbbb140569b4255cebb9669562588119"
server_key_version: "encryptionkey@0"
I20260810 14:37:23.150170 21755 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260810 14:37:23.154808 21785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:23.157392 21755 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.003s
I20260810 14:37:23.157670 21755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "2fdae4323510473eb4a2fbeca78516bd"
format_stamp: "Formatted at 2026-08-10 14:37:23 on dist-test-slave-h6n0"
server_key: "17d29cb82ae71b04677c53f4312c5083"
server_key_iv: "dbbb140569b4255cebb9669562588119"
server_key_version: "encryptionkey@0"
I20260810 14:37:23.157902 21755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:23.179378 21755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:23.184942 21755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:23.185277 21755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:23.207065 21755 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:38057
I20260810 14:37:23.207085 21867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:38057 every 8 connection(s)
I20260810 14:37:23.208639 21755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:37:23.212659 21868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:23.218907 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 21755
I20260810 14:37:23.219343 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:37:23.221496 21725 external_mini_cluster.cc:1428] Setting key 3df8b69200cd312e4d5679de1b067aa9
I20260810 14:37:23.223462 21868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd: Bootstrap starting.
I20260810 14:37:23.227005 21868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:23.228039 21868 log.cc:824] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:23.231170 21868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd: No bootstrap required, opened a new log
I20260810 14:37:23.240928 21868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdae4323510473eb4a2fbeca78516bd" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38057 } }
I20260810 14:37:23.241423 21868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:23.241539 21868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdae4323510473eb4a2fbeca78516bd, State: Initialized, Role: FOLLOWER
I20260810 14:37:23.242141 21868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdae4323510473eb4a2fbeca78516bd" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38057 } }
I20260810 14:37:23.242340 21868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:37:23.242565 21868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:37:23.242748 21868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:23.244966 21868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdae4323510473eb4a2fbeca78516bd" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38057 } }
I20260810 14:37:23.245599 21868 leader_election.cc:302] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fdae4323510473eb4a2fbeca78516bd; no voters:
I20260810 14:37:23.246649 21868 leader_election.cc:288] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:37:23.246876 21879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:23.248769 21879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [term 1 LEADER]: Becoming Leader. State: Replica: 2fdae4323510473eb4a2fbeca78516bd, State: Running, Role: LEADER
I20260810 14:37:23.249437 21879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdae4323510473eb4a2fbeca78516bd" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38057 } }
I20260810 14:37:23.249747 21868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:37:23.252328 21883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fdae4323510473eb4a2fbeca78516bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdae4323510473eb4a2fbeca78516bd" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38057 } } }
I20260810 14:37:23.252739 21883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:23.256983 21885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fdae4323510473eb4a2fbeca78516bd. Latest consensus state: current_term: 1 leader_uuid: "2fdae4323510473eb4a2fbeca78516bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fdae4323510473eb4a2fbeca78516bd" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38057 } } }
I20260810 14:37:23.257249 21885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd [sys.catalog]: This master's current role is: LEADER
Aug 10 14:37:23 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:23.260857 21892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:37:23.268275 21892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:37:23.275347 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:23.225932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33418 (local address 127.21.55.126:38057)
0810 14:37:23.227009 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:23.227025 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:23.228598 (+ 1573us) server_negotiation.cc:407] Connection header received
0810 14:37:23.240200 (+ 11602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:23.240233 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:23.240729 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:23.241223 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:23.244858 (+ 3635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:23.247472 (+ 2614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:23.250671 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:23.251857 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:23.254292 (+ 2435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:23.261230 (+ 6938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:23.261247 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:23.261278 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:23.261344 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:23.266732 (+ 5388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:23.267819 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:23.267829 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:23.267840 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:23.268117 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:23.268989 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:23.268997 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:23.269001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:23.269913 (+ 912us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:23.270253 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:23.273380 (+ 3127us) server_negotiation.cc:299] Negotiation successful
0810 14:37:23.273771 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":814,"thread_start_us":544,"threads_started":1}
I20260810 14:37:23.281119 21892 catalog_manager.cc:1347] Generated new cluster ID: 836c9bc184524c5cb5dfc4985f233b71
I20260810 14:37:23.281244 21892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:37:23.281798 21912 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 14:37:23.290818 21892 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:37:23.292706 21892 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:37:23.300077 21892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fdae4323510473eb4a2fbeca78516bd: Generated new TSK 0
I20260810 14:37:23.301083 21892 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:37:23.354431 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38057
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:23.724047 21918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:23.724622 21918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:23.724879 21918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:23.732890 21918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:23.733019 21918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:23.733155 21918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:37:23.743139 21918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38057
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:23.744385 21918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:23.745784 21918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:23.760337 21924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:23.760432 21925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:23.760710 21927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:23.761266 21918 server_base.cc:1034] running on GCE node
I20260810 14:37:23.761934 21918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:23.763190 21918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:23.764483 21918 hybrid_clock.cc:648] HybridClock initialized: now 1786372643764433 us; error 37 us; skew 500 ppm
I20260810 14:37:23.764808 21918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:23 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372643, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:23.769236 21918 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:37:23.771158 21918 webserver.cc:460] Webserver started at http://127.21.55.65:39739/ using document root <none> and password file <none>
I20260810 14:37:23.771883 21918 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:23.772048 21918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:23.772382 21918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:23.775669 21918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "bdbf1c05037f4f85ab719472a9395a62"
format_stamp: "Formatted at 2026-08-10 14:37:23 on dist-test-slave-h6n0"
server_key: "4f9b120a92c48191d9dd050dbbc08d69"
server_key_iv: "a9edc4005025d91cf04ecf9202f46f4a"
server_key_version: "encryptionkey@0"
I20260810 14:37:23.776861 21918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "bdbf1c05037f4f85ab719472a9395a62"
format_stamp: "Formatted at 2026-08-10 14:37:23 on dist-test-slave-h6n0"
server_key: "4f9b120a92c48191d9dd050dbbc08d69"
server_key_iv: "a9edc4005025d91cf04ecf9202f46f4a"
server_key_version: "encryptionkey@0"
I20260810 14:37:23.783695 21918 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 14:37:23.787210 21939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:23.788590 21918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 14:37:23.788779 21918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "bdbf1c05037f4f85ab719472a9395a62"
format_stamp: "Formatted at 2026-08-10 14:37:23 on dist-test-slave-h6n0"
server_key: "4f9b120a92c48191d9dd050dbbc08d69"
server_key_iv: "a9edc4005025d91cf04ecf9202f46f4a"
server_key_version: "encryptionkey@0"
I20260810 14:37:23.788993 21918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:23.807305 21918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:23.811708 21918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:23.812008 21918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:23.813138 21918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:23.814679 21918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:23.814781 21918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:23.814961 21918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:23.815040 21918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:23.860371 21918 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:41445
I20260810 14:37:23.860463 22129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:41445 every 8 connection(s)
I20260810 14:37:23.861794 21918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:37:23.868007 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 21918
I20260810 14:37:23.868292 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:37:23.869153 21725 external_mini_cluster.cc:1428] Setting key 65b13820b8eeabbbf3f72f2791eaa743
Aug 10 14:37:23 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372643, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:23.886821 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:23.864726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:57845 (local address 127.21.55.126:38057)
0810 14:37:23.865227 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:23.865232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:23.866040 (+ 808us) server_negotiation.cc:407] Connection header received
0810 14:37:23.868604 (+ 2564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:23.868609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:23.868691 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:23.868810 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:23.870892 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:23.871691 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:23.874138 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:23.874631 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:23.874740 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:23.879209 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:23.879230 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:23.879242 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:23.879295 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:23.882907 (+ 3612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:23.883654 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:23.883662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:23.883669 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:23.883818 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:23.884264 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:23.884272 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:23.884278 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:23.884662 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:23.884825 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:23.886041 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0810 14:37:23.886287 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":274,"threads_started":1}
I20260810 14:37:23.888110 22136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:23.865111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38057 (local address 127.21.55.65:57845)
0810 14:37:23.865865 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:23.865918 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:23.868274 (+ 2356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:23.869088 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:23.869100 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:23.869987 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:23.870676 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:23.870702 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:23.871853 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:23.871858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:23.873958 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:23.873976 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:23.874893 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:23.874903 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:23.875265 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:23.876062 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:23.876103 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:23.878932 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:23.883070 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:23.883077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:23.883088 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:23.883512 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:23.883941 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:23.883944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:23.883947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:23.884129 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:23.884938 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:23.884949 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:23.885484 (+ 535us) client_negotiation.cc:769] Sending connection context
0810 14:37:23.885868 (+ 384us) client_negotiation.cc:240] Negotiation successful
0810 14:37:23.886423 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":327,"threads_started":1}
I20260810 14:37:23.890178 22132 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38057
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
I20260810 14:37:23.890604 22132 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:23.891453 22132 heartbeater.cc:511] Master 127.21.55.126:38057 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
I20260810 14:37:23.893985 21809 ts_manager.cc:194] Registered new tserver with Master: bdbf1c05037f4f85ab719472a9395a62 (127.21.55.65:41445)
I20260810 14:37:23.895673 21809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:57845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:37:23.914417 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38057
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:24.272905 22141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:24.273221 22141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:24.273412 22141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:24.284372 22141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:24.284523 22141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:24.284715 22141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:37:24.295295 22141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38057
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:24.296588 22141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:24.297917 22141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:24.312462 22155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:24.312666 22153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:24.312943 22157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:24.314204 22141 server_base.cc:1034] running on GCE node
I20260810 14:37:24.314896 22141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:24.315831 22141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:24.317381 22141 hybrid_clock.cc:648] HybridClock initialized: now 1786372644317325 us; error 97 us; skew 500 ppm
I20260810 14:37:24.317924 22141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:24 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372644, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:24.321336 22141 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:37:24.322964 22141 webserver.cc:460] Webserver started at http://127.21.55.66:34847/ using document root <none> and password file <none>
I20260810 14:37:24.323675 22141 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:24.323827 22141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:24.324164 22141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:24.327116 22141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "7b6bdbe64f474636b25e068b91975703"
format_stamp: "Formatted at 2026-08-10 14:37:24 on dist-test-slave-h6n0"
server_key: "bc11d02c162747f72e69390e2465bd9b"
server_key_iv: "7076c7e1131d53e55634119cdb5c34b3"
server_key_version: "encryptionkey@0"
I20260810 14:37:24.327899 22141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "7b6bdbe64f474636b25e068b91975703"
format_stamp: "Formatted at 2026-08-10 14:37:24 on dist-test-slave-h6n0"
server_key: "bc11d02c162747f72e69390e2465bd9b"
server_key_iv: "7076c7e1131d53e55634119cdb5c34b3"
server_key_version: "encryptionkey@0"
I20260810 14:37:24.333202 22141 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:37:24.337536 22170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:24.339109 22141 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:37:24.339298 22141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "7b6bdbe64f474636b25e068b91975703"
format_stamp: "Formatted at 2026-08-10 14:37:24 on dist-test-slave-h6n0"
server_key: "bc11d02c162747f72e69390e2465bd9b"
server_key_iv: "7076c7e1131d53e55634119cdb5c34b3"
server_key_version: "encryptionkey@0"
I20260810 14:37:24.339521 22141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:24.368940 22141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:24.373991 22141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:24.374325 22141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:24.375460 22141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:24.376914 22141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:24.377028 22141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:24.377184 22141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:24.377260 22141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:24.416858 22141 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:37215
I20260810 14:37:24.416903 22358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:37215 every 8 connection(s)
I20260810 14:37:24.418136 22141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:37:24.424655 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 22141
I20260810 14:37:24.424934 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:37:24.425640 21725 external_mini_cluster.cc:1428] Setting key 963bfa063c0d6ddd044313240e4f97b1
Aug 10 14:37:24 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372644, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:24.442955 22362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:24.422826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:40799 (local address 127.21.55.126:38057)
0810 14:37:24.423393 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:24.423401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:24.423558 (+ 157us) server_negotiation.cc:407] Connection header received
0810 14:37:24.425639 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:24.425645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:24.425739 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:24.425878 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:24.428002 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:24.429123 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:24.430803 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:24.431614 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:24.431764 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:24.435318 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:24.435328 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:24.435336 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:24.435379 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:24.439177 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:24.439942 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:24.439951 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:24.439957 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:24.440133 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:24.440631 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:24.440639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:24.440645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:24.441069 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:24.441239 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:24.442140 (+ 901us) server_negotiation.cc:299] Negotiation successful
0810 14:37:24.442400 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":276,"threads_started":1}
I20260810 14:37:24.443607 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:24.422006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38057 (local address 127.21.55.66:40799)
0810 14:37:24.423240 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:24.423289 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:24.425365 (+ 2076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:24.426191 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:24.426202 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:24.426991 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:24.427820 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:24.427843 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:24.429299 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:24.429303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:24.430642 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:24.430662 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:24.431821 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:24.431826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:24.432032 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:24.432740 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:24.432762 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:24.435069 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:24.439340 (+ 4271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:24.439347 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:24.439358 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:24.439785 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:24.440273 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:24.440277 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:24.440283 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:24.440495 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:24.441394 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:24.441400 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:24.441709 (+ 309us) client_negotiation.cc:769] Sending connection context
0810 14:37:24.441963 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 14:37:24.442362 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"thread_start_us":569,"threads_started":1}
I20260810 14:37:24.445313 22359 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38057
I20260810 14:37:24.445660 22359 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
I20260810 14:37:24.446497 22359 heartbeater.cc:511] Master 127.21.55.126:38057 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
I20260810 14:37:24.448571 21809 ts_manager.cc:194] Registered new tserver with Master: 7b6bdbe64f474636b25e068b91975703 (127.21.55.66:37215)
I20260810 14:37:24.450318 21809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:40799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:37:24.473328 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38057
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:24.849278 22366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:24.849644 22366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:24.849879 22366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:24.857277 22366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:24.857434 22366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:24.857635 22366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:37:24.875104 22366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38057
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:24.876617 22366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:24.878471 22366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:24.894198 22375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:24.894208 22379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:24.894237 22376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:24.895462 22366 server_base.cc:1034] running on GCE node
I20260810 14:37:24.896195 22366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:24.897270 22366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:24.898589 22366 hybrid_clock.cc:648] HybridClock initialized: now 1786372644898501 us; error 39 us; skew 500 ppm
I20260810 14:37:24.898806 22132 heartbeater.cc:503] Master 127.21.55.126:38057 was elected leader, sending a full tablet report...
I20260810 14:37:24.899008 22366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:24 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372644, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:24.902765 22366 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:37:24.904317 22366 webserver.cc:460] Webserver started at http://127.21.55.67:46563/ using document root <none> and password file <none>
I20260810 14:37:24.904958 22366 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:24.905074 22366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:24.905328 22366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:24.908456 22366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "58b6115059dd4349a09d7a7aad6e664b"
format_stamp: "Formatted at 2026-08-10 14:37:24 on dist-test-slave-h6n0"
server_key: "418b3967972c2273b2e04e9e631466a2"
server_key_iv: "28d5506659e1a48db7b2466109ae5357"
server_key_version: "encryptionkey@0"
I20260810 14:37:24.909202 22366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "58b6115059dd4349a09d7a7aad6e664b"
format_stamp: "Formatted at 2026-08-10 14:37:24 on dist-test-slave-h6n0"
server_key: "418b3967972c2273b2e04e9e631466a2"
server_key_iv: "28d5506659e1a48db7b2466109ae5357"
server_key_version: "encryptionkey@0"
I20260810 14:37:24.916297 22366 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260810 14:37:24.920473 22393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:24.922003 22366 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 14:37:24.922189 22366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "58b6115059dd4349a09d7a7aad6e664b"
format_stamp: "Formatted at 2026-08-10 14:37:24 on dist-test-slave-h6n0"
server_key: "418b3967972c2273b2e04e9e631466a2"
server_key_iv: "28d5506659e1a48db7b2466109ae5357"
server_key_version: "encryptionkey@0"
I20260810 14:37:24.922406 22366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:24.945993 22366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:24.951475 22366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:24.951804 22366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:24.952965 22366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:24.954510 22366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:24.954670 22366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260810 14:37:24.954802 22366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:24.954874 22366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:24.994499 22366 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:37985
I20260810 14:37:24.994750 22551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:37985 every 8 connection(s)
I20260810 14:37:24.995898 22366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:37:25.004170 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 22366
I20260810 14:37:25.004405 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:37:25.004961 21725 external_mini_cluster.cc:1428] Setting key 6ba1134dbd06085998ca64b4493e4c88
Aug 10 14:37:25 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372644, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:25.018354 22555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:24.998720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:35099 (local address 127.21.55.126:38057)
0810 14:37:24.999237 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:24.999244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:25.000154 (+ 910us) server_negotiation.cc:407] Connection header received
0810 14:37:25.001984 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:25.001991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:25.002070 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:25.002176 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:25.003784 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.005024 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.007038 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.007672 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.007790 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:25.012318 (+ 4528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:25.012323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:25.012326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:25.012359 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:25.015126 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.016194 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.016198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.016200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.016332 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.016813 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.016817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.016818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.017090 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:25.017201 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:25.017909 (+ 708us) server_negotiation.cc:299] Negotiation successful
0810 14:37:25.018080 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":247,"threads_started":1}
I20260810 14:37:25.019526 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:24.999221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38057 (local address 127.21.55.67:35099)
0810 14:37:24.999990 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:25.000027 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:25.001707 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:25.002391 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:25.002399 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:25.003027 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:25.003574 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:25.003591 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.005226 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.005231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:25.006869 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:25.006883 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.007952 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.007960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:25.008321 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:25.009655 (+ 1334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:25.009686 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:25.012089 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:25.015402 (+ 3313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:25.015414 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:25.015433 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:25.016050 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:25.016465 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:25.016469 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:25.016472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:25.016699 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:25.017214 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:25.017220 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:25.017558 (+ 338us) client_negotiation.cc:769] Sending connection context
0810 14:37:25.017807 (+ 249us) client_negotiation.cc:240] Negotiation successful
0810 14:37:25.018229 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":304,"threads_started":1}
I20260810 14:37:25.021625 22552 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38057
I20260810 14:37:25.022182 22552 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:25.023103 22552 heartbeater.cc:511] Master 127.21.55.126:38057 requested a full tablet report, sending...
I20260810 14:37:25.024844 21809 ts_manager.cc:194] Registered new tserver with Master: 58b6115059dd4349a09d7a7aad6e664b (127.21.55.67:37985)
I20260810 14:37:25.025868 21809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:35099
I20260810 14:37:25.036095 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:37:25 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372645, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:37:25 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372645, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:37:25.088392 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:25.070618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43594 (local address 127.21.55.65:41445)
0810 14:37:25.071626 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:25.071658 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:25.071704 (+ 46us) server_negotiation.cc:407] Connection header received
0810 14:37:25.072008 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:25.072018 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:25.072398 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:25.072740 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:25.075225 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.076274 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.077578 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.078153 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.078310 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:25.082403 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:25.082414 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:25.082429 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:25.082462 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:25.085453 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.086202 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.086210 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.086212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.086338 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.086915 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.086918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.086920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.087254 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:25.087498 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:25.087763 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 14:37:25.087980 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":440,"threads_started":1}
W20260810 14:37:25.089735 21991 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43594
Aug 10 14:37:25 dist-test-slave-h6n0 krb5kdc[21736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:37:25 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:37:25.139182 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:25.124116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43598 (local address 127.21.55.65:41445)
0810 14:37:25.124375 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:25.124382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:25.124625 (+ 243us) server_negotiation.cc:407] Connection header received
0810 14:37:25.124851 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:25.124856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:25.124934 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:25.125045 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:25.126455 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.127217 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.128501 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.128932 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.129071 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:25.133549 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:25.133555 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:25.133558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:25.133591 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:25.136464 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.137246 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.137251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.137253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.137387 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.137820 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.137824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.137826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.138080 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:25.138259 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:25.138743 (+ 484us) server_negotiation.cc:299] Negotiation successful
0810 14:37:25.138923 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 14:37:25.141117 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 21918
I20260810 14:37:25.155972 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:25.142285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43602 (local address 127.21.55.65:41445)
0810 14:37:25.142523 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:25.142553 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:25.142794 (+ 241us) server_negotiation.cc:407] Connection header received
0810 14:37:25.143003 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:25.143007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:25.143067 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:25.143177 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:25.144593 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.145346 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.147158 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.147911 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.148124 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:25.150838 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:25.150844 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:25.150847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:25.150876 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:25.153348 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.154019 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.154024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.154026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.154156 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.154574 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.154585 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.154588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.154837 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:25.155099 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:25.155401 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 14:37:25.155621 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 14:37:25.157120 22122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:25.256748 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 21918
I20260810 14:37:25.289372 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22141
Aug 10 14:37:25 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:37:25.308444 22572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:25.290828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.21.55.66:37215)
0810 14:37:25.291407 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:25.291414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:25.291440 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:37:25.291504 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:25.291510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:25.291749 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:25.291934 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:25.293247 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.294684 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.296006 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.296573 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.296730 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:25.301315 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:25.301327 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:25.301341 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:25.301375 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:25.304729 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.305711 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.305718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.305720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.305880 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.306375 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.306383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.306386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.306886 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:25.307258 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:25.307554 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 14:37:25.307901 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":314,"threads_started":1}
I20260810 14:37:25.310187 22349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:25.402657 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 22141
I20260810 14:37:25.429852 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22366
Aug 10 14:37:25 dist-test-slave-h6n0 krb5kdc[21736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:37:25.448726 22574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:25.430956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.21.55.67:37985)
0810 14:37:25.431526 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:25.431534 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:25.431565 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:37:25.431627 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:25.431633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:25.431848 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:25.432011 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:25.433343 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.435000 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.436423 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:25.437057 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:25.437171 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:25.441526 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:25.441539 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:25.441565 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:25.441602 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:25.444678 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.445510 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.445519 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.445522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.445659 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:25.446125 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:25.446132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:25.446135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:25.446856 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:25.447234 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:25.447835 (+ 601us) server_negotiation.cc:299] Negotiation successful
0810 14:37:25.448387 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":271,"threads_started":1}
I20260810 14:37:25.450506 22547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:25.540365 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 22366
I20260810 14:37:25.567237 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 21755
I20260810 14:37:25.568955 21862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:25.680354 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 21755
2026-08-10T14:37:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4543 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:37:25 dist-test-slave-h6n0 krb5kdc[22584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:37:25 dist-test-slave-h6n0 krb5kdc[22584](info): set up 2 sockets
Aug 10 14:37:25 dist-test-slave-h6n0 krb5kdc[22584](info): commencing operation
krb5kdc: starting...
W20260810 14:37:27.035593 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:37:27 dist-test-slave-h6n0 krb5kdc[22584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:37:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:37:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:37:27.246308 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:37319
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:37319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:27.612879 22601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:27.613171 22601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:27.613335 22601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:27.621332 22601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:37:27.621454 22601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:27.621553 22601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:27.621642 22601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:37:27.621712 22601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:37:27.632540 22601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:37319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:37319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:27.634878 22601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:27.636276 22601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:27.648487 22607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:27.648439 22608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:27.648783 22610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:27.649924 22601 server_base.cc:1034] running on GCE node
I20260810 14:37:27.651101 22601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:27.652853 22601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:27.654491 22601 hybrid_clock.cc:648] HybridClock initialized: now 1786372647654143 us; error 40 us; skew 500 ppm
I20260810 14:37:27.655273 22601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:27 dist-test-slave-h6n0 krb5kdc[22584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372647, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:27.659332 22601 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:37:27.661037 22601 webserver.cc:460] Webserver started at http://127.21.55.126:41165/ using document root <none> and password file <none>
I20260810 14:37:27.661751 22601 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:27.661860 22601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:27.662189 22601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:27.665217 22601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "2aad4a778050417da18454f61e3484c6"
format_stamp: "Formatted at 2026-08-10 14:37:27 on dist-test-slave-h6n0"
server_key: "74bc1ab5a2627ee4260c8a3b608b80e2"
server_key_iv: "e220d88ca658a9785b83e8b7abca99c6"
server_key_version: "encryptionkey@0"
I20260810 14:37:27.665982 22601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "2aad4a778050417da18454f61e3484c6"
format_stamp: "Formatted at 2026-08-10 14:37:27 on dist-test-slave-h6n0"
server_key: "74bc1ab5a2627ee4260c8a3b608b80e2"
server_key_iv: "e220d88ca658a9785b83e8b7abca99c6"
server_key_version: "encryptionkey@0"
I20260810 14:37:27.671324 22601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:37:27.674839 22624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:27.676397 22601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:37:27.676600 22601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "2aad4a778050417da18454f61e3484c6"
format_stamp: "Formatted at 2026-08-10 14:37:27 on dist-test-slave-h6n0"
server_key: "74bc1ab5a2627ee4260c8a3b608b80e2"
server_key_iv: "e220d88ca658a9785b83e8b7abca99c6"
server_key_version: "encryptionkey@0"
I20260810 14:37:27.676806 22601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:27.702189 22601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:27.706763 22601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:27.707127 22601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:27.731369 22601 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:37319
I20260810 14:37:27.731370 22691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:37319 every 8 connection(s)
I20260810 14:37:27.732906 22601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:37:27.736837 22692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:27.739939 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 22601
I20260810 14:37:27.740209 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:37:27.741022 21725 external_mini_cluster.cc:1428] Setting key 5e96309f884854ce0c26a0114aa1aac8
I20260810 14:37:27.754914 22692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6: Bootstrap starting.
I20260810 14:37:27.758198 22692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:27.759292 22692 log.cc:824] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:37:27 dist-test-slave-h6n0 krb5kdc[22584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:27.763293 22692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6: No bootstrap required, opened a new log
I20260810 14:37:27.770879 22697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:27.744173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59822 (local address 127.21.55.126:37319)
0810 14:37:27.747523 (+ 3350us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:27.747542 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:27.747595 (+ 53us) server_negotiation.cc:407] Connection header received
0810 14:37:27.750014 (+ 2419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:27.750038 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:27.750436 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:27.751013 (+ 577us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:27.752812 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:27.754144 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:27.755788 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:27.756555 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:27.756794 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:27.762433 (+ 5639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:27.762441 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:27.762455 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:27.762492 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:27.765849 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:27.767097 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:27.767108 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:27.767113 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:27.767282 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:27.767953 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:27.767957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:27.767959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:27.768506 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:27.768757 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:27.769338 (+ 581us) server_negotiation.cc:299] Negotiation successful
0810 14:37:27.769757 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2685,"thread_start_us":2426,"threads_started":1}
I20260810 14:37:27.773335 22692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aad4a778050417da18454f61e3484c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37319 } }
I20260810 14:37:27.773756 22692 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:27.773885 22692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aad4a778050417da18454f61e3484c6, State: Initialized, Role: FOLLOWER
I20260810 14:37:27.774503 22692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aad4a778050417da18454f61e3484c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37319 } }
I20260810 14:37:27.774819 22692 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:37:27.774988 22692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:37:27.775164 22692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:27.777303 22692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aad4a778050417da18454f61e3484c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37319 } }
I20260810 14:37:27.777805 22692 leader_election.cc:302] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2aad4a778050417da18454f61e3484c6; no voters:
I20260810 14:37:27.778849 22692 leader_election.cc:288] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:37:27.779222 22700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:27.780838 22700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [term 1 LEADER]: Becoming Leader. State: Replica: 2aad4a778050417da18454f61e3484c6, State: Running, Role: LEADER
I20260810 14:37:27.781605 22692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:37:27.781665 22700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aad4a778050417da18454f61e3484c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37319 } }
I20260810 14:37:27.785805 22704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2aad4a778050417da18454f61e3484c6. Latest consensus state: current_term: 1 leader_uuid: "2aad4a778050417da18454f61e3484c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aad4a778050417da18454f61e3484c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37319 } } }
I20260810 14:37:27.785753 22701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2aad4a778050417da18454f61e3484c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aad4a778050417da18454f61e3484c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37319 } } }
I20260810 14:37:27.786269 22704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:27.786285 22701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:27.789453 22716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:37:27.793887 22716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:37:27.804122 22716 catalog_manager.cc:1347] Generated new cluster ID: a3f109af6139489f8d56a990541b0384
I20260810 14:37:27.804271 22716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:37:27.821998 22716 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:37:27.823482 22716 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:37:27.832623 22716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2aad4a778050417da18454f61e3484c6: Generated new TSK 0
I20260810 14:37:27.833956 22716 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:37:27.895020 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37319
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:28.265374 22736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:28.265877 22736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:28.266063 22736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:28.278510 22736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:28.278743 22736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:28.279036 22736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:37:28.292428 22736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37319
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:28.293751 22736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:28.295194 22736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:28.311621 22743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:28.311712 22744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:28.311712 22746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:28.313486 22736 server_base.cc:1034] running on GCE node
I20260810 14:37:28.314667 22736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:28.315749 22736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:28.317144 22736 hybrid_clock.cc:648] HybridClock initialized: now 1786372648316994 us; error 111 us; skew 500 ppm
I20260810 14:37:28.317577 22736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:28 dist-test-slave-h6n0 krb5kdc[22584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372648, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:28.321596 22736 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:37:28.323328 22736 webserver.cc:460] Webserver started at http://127.21.55.65:44193/ using document root <none> and password file <none>
I20260810 14:37:28.324008 22736 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:28.324285 22736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:28.324637 22736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:28.328198 22736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb"
format_stamp: "Formatted at 2026-08-10 14:37:28 on dist-test-slave-h6n0"
server_key: "09312a973740d923fdfea457b1283f24"
server_key_iv: "0dc8ae9d4b91dddbacac232f082961c9"
server_key_version: "encryptionkey@0"
I20260810 14:37:28.329303 22736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb"
format_stamp: "Formatted at 2026-08-10 14:37:28 on dist-test-slave-h6n0"
server_key: "09312a973740d923fdfea457b1283f24"
server_key_iv: "0dc8ae9d4b91dddbacac232f082961c9"
server_key_version: "encryptionkey@0"
I20260810 14:37:28.336220 22736 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 14:37:28.339903 22754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:28.341478 22736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:37:28.341702 22736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb"
format_stamp: "Formatted at 2026-08-10 14:37:28 on dist-test-slave-h6n0"
server_key: "09312a973740d923fdfea457b1283f24"
server_key_iv: "0dc8ae9d4b91dddbacac232f082961c9"
server_key_version: "encryptionkey@0"
I20260810 14:37:28.341939 22736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:28.372192 22736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:28.377426 22736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:28.377869 22736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:28.379087 22736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:28.380718 22736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:28.380841 22736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:28.381009 22736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:28.381093 22736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:28.433066 22736 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:45165
I20260810 14:37:28.433178 22932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:45165 every 8 connection(s)
I20260810 14:37:28.434446 22736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:37:28.440431 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 22736
I20260810 14:37:28.440701 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:37:28.441520 21725 external_mini_cluster.cc:1428] Setting key 231b00bd1d6af309d7d48e7d9b02150e
Aug 10 14:37:28 dist-test-slave-h6n0 krb5kdc[22584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372648, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:28.465310 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:28.438119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:38267 (local address 127.21.55.126:37319)
0810 14:37:28.438842 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:28.438846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:28.439674 (+ 828us) server_negotiation.cc:407] Connection header received
0810 14:37:28.442584 (+ 2910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:28.442594 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:28.442684 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:28.442783 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:28.445265 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:28.446084 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:28.449478 (+ 3394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:28.450419 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:28.450633 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:28.456104 (+ 5471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:28.456115 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:28.456123 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:28.456182 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:28.460816 (+ 4634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:28.461848 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:28.461855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:28.461859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:28.462025 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:28.462610 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:28.462618 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:28.462624 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:28.463106 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:28.463295 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:28.464574 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0810 14:37:28.464840 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":406,"threads_started":1}
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
I20260810 14:37:28.466843 22938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:28.438439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:37319 (local address 127.21.55.65:38267)
0810 14:37:28.439435 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:28.439495 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:28.442166 (+ 2671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:28.443431 (+ 1265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:28.443442 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:28.444191 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:28.445049 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:28.445077 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:28.446273 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:28.446284 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:28.449126 (+ 2842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:28.449148 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:28.450855 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:28.450868 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:28.451369 (+ 501us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:28.452893 (+ 1524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:28.452927 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:28.455806 (+ 2879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:28.461023 (+ 5217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:28.461032 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:28.461048 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:28.461690 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:28.462217 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:28.462221 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:28.462223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:28.462432 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:28.463380 (+ 948us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:28.463391 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:28.463996 (+ 605us) client_negotiation.cc:769] Sending connection context
0810 14:37:28.464376 (+ 380us) client_negotiation.cc:240] Negotiation successful
0810 14:37:28.465039 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":408,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
I20260810 14:37:28.469151 22933 heartbeater.cc:347] Connected to a master server at 127.21.55.126:37319
I20260810 14:37:28.469550 22933 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:28.470994 22933 heartbeater.cc:511] Master 127.21.55.126:37319 requested a full tablet report, sending...
I20260810 14:37:28.473520 22646 ts_manager.cc:194] Registered new tserver with Master: 6f0ed51f3ba9416b9f2c78ca56b498fb (127.21.55.65:45165)
I20260810 14:37:28.475088 22646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:38267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:37:28.499120 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37319
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:28.890147 22943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:28.890640 22943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:28.890846 22943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:28.900359 22943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:28.900552 22943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:28.900764 22943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:37:28.912793 22943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37319
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:28.914232 22943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:28.915736 22943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:28.930814 22956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:28.931288 22958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:28.931826 22961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:28.933646 22943 server_base.cc:1034] running on GCE node
I20260810 14:37:28.934830 22943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:28.936345 22943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:28.937686 22943 hybrid_clock.cc:648] HybridClock initialized: now 1786372648937625 us; error 74 us; skew 500 ppm
I20260810 14:37:28.938133 22943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:28 dist-test-slave-h6n0 krb5kdc[22584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372648, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:28.942427 22943 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:37:28.945143 22943 webserver.cc:460] Webserver started at http://127.21.55.66:44197/ using document root <none> and password file <none>
I20260810 14:37:28.946486 22943 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:28.946696 22943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:28.947064 22943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:28.950857 22943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "2cc65c7143a04891be5c8a27536e4748"
format_stamp: "Formatted at 2026-08-10 14:37:28 on dist-test-slave-h6n0"
server_key: "4d6e8ed43c0897828c53b5c1afcbc5f8"
server_key_iv: "bfbc4e1a38768bc8e950681a83be5bc2"
server_key_version: "encryptionkey@0"
I20260810 14:37:28.951730 22943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "2cc65c7143a04891be5c8a27536e4748"
format_stamp: "Formatted at 2026-08-10 14:37:28 on dist-test-slave-h6n0"
server_key: "4d6e8ed43c0897828c53b5c1afcbc5f8"
server_key_iv: "bfbc4e1a38768bc8e950681a83be5bc2"
server_key_version: "encryptionkey@0"
I20260810 14:37:28.957763 22943 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260810 14:37:28.962772 22969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:28.964672 22943 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:37:28.964881 22943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "2cc65c7143a04891be5c8a27536e4748"
format_stamp: "Formatted at 2026-08-10 14:37:28 on dist-test-slave-h6n0"
server_key: "4d6e8ed43c0897828c53b5c1afcbc5f8"
server_key_iv: "bfbc4e1a38768bc8e950681a83be5bc2"
server_key_version: "encryptionkey@0"
I20260810 14:37:28.965155 22943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:28.995872 22943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:29.002446 22943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:29.002853 22943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:29.004079 22943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:29.005686 22943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:29.005781 22943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:29.005921 22943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:29.005976 22943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:29.047755 22943 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:35803
I20260810 14:37:29.047813 23133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:35803 every 8 connection(s)
I20260810 14:37:29.049151 22943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:37:29.056566 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 22943
I20260810 14:37:29.056826 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:37:29.057332 21725 external_mini_cluster.cc:1428] Setting key 6744a4fe1622bda8a6799feb85e1efd2
Aug 10 14:37:29 dist-test-slave-h6n0 krb5kdc[22584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372648, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:29.074412 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:29.052051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:46349 (local address 127.21.55.126:37319)
0810 14:37:29.052578 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:29.052583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:29.053699 (+ 1116us) server_negotiation.cc:407] Connection header received
0810 14:37:29.055857 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:29.055863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:29.055968 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:29.056132 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:29.057941 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.058864 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.060827 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.061797 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.062026 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:29.066257 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:29.066267 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:29.066274 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:29.066325 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:29.069882 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:29.071048 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:29.071053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:29.071055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:29.071194 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:29.071934 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:29.071941 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:29.071944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:29.072441 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:29.072619 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:29.073763 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0810 14:37:29.074010 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":276,"threads_started":1}
I20260810 14:37:29.075989 23138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:29.052650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:37319 (local address 127.21.55.66:46349)
0810 14:37:29.053416 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:29.053470 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:29.055568 (+ 2098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:29.056360 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:29.056369 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:29.057085 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:29.057749 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:29.057769 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.059061 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.059065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:29.060586 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:29.060605 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.062305 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.062342 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:29.062685 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:29.063545 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:29.063571 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:29.065976 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:29.070029 (+ 4053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:29.070040 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:29.070061 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:29.070908 (+ 847us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:29.071328 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:29.071337 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:29.071345 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:29.071645 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:29.072653 (+ 1008us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:29.072665 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:29.073216 (+ 551us) client_negotiation.cc:769] Sending connection context
0810 14:37:29.073587 (+ 371us) client_negotiation.cc:240] Negotiation successful
0810 14:37:29.074234 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":303,"threads_started":1}
I20260810 14:37:29.077690 23134 heartbeater.cc:347] Connected to a master server at 127.21.55.126:37319
I20260810 14:37:29.078076 23134 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:29.078981 23134 heartbeater.cc:511] Master 127.21.55.126:37319 requested a full tablet report, sending...
I20260810 14:37:29.080991 22646 ts_manager.cc:194] Registered new tserver with Master: 2cc65c7143a04891be5c8a27536e4748 (127.21.55.66:35803)
I20260810 14:37:29.082243 22646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:46349
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:37:29.110013 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37319
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:36869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:37:29.478622 22933 heartbeater.cc:503] Master 127.21.55.126:37319 was elected leader, sending a full tablet report...
W20260810 14:37:29.482199 23143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:29.482513 23143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:29.482705 23143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:29.490429 23143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:29.490579 23143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:29.490721 23143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:37:29.501111 23143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:36869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37319
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:29.503001 23143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:29.505076 23143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:29.519205 23148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:29.519256 23153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:29.519222 23149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:29.520478 23143 server_base.cc:1034] running on GCE node
I20260810 14:37:29.521138 23143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:29.522131 23143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:29.523397 23143 hybrid_clock.cc:648] HybridClock initialized: now 1786372649523355 us; error 59 us; skew 500 ppm
I20260810 14:37:29.523753 23143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:29 dist-test-slave-h6n0 krb5kdc[22584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372649, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:29.527446 23143 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:37:29.529150 23143 webserver.cc:460] Webserver started at http://127.21.55.67:45197/ using document root <none> and password file <none>
I20260810 14:37:29.529841 23143 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:29.529999 23143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:29.530321 23143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:29.533404 23143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "b3a8327cd8e0459d85acafd0e21522af"
format_stamp: "Formatted at 2026-08-10 14:37:29 on dist-test-slave-h6n0"
server_key: "4d55f03da2dbe1eb9789e6374e09e4e2"
server_key_iv: "6481aee0ce605d27358c09fdc299265d"
server_key_version: "encryptionkey@0"
I20260810 14:37:29.534217 23143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "b3a8327cd8e0459d85acafd0e21522af"
format_stamp: "Formatted at 2026-08-10 14:37:29 on dist-test-slave-h6n0"
server_key: "4d55f03da2dbe1eb9789e6374e09e4e2"
server_key_iv: "6481aee0ce605d27358c09fdc299265d"
server_key_version: "encryptionkey@0"
I20260810 14:37:29.540410 23143 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.002s
I20260810 14:37:29.545038 23163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:29.547310 23143 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:37:29.547541 23143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "b3a8327cd8e0459d85acafd0e21522af"
format_stamp: "Formatted at 2026-08-10 14:37:29 on dist-test-slave-h6n0"
server_key: "4d55f03da2dbe1eb9789e6374e09e4e2"
server_key_iv: "6481aee0ce605d27358c09fdc299265d"
server_key_version: "encryptionkey@0"
I20260810 14:37:29.547775 23143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:29.564613 23143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:29.569327 23143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:29.569733 23143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:29.570865 23143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:29.572285 23143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:29.572366 23143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:29.572508 23143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:29.572577 23143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:29.612542 23143 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:43609
I20260810 14:37:29.612819 23343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:43609 every 8 connection(s)
I20260810 14:37:29.613935 23143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:37:29.620896 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23143
I20260810 14:37:29.621133 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:37:29.621595 21725 external_mini_cluster.cc:1428] Setting key 677fda1788f1cbc1bda3cc1d6423cec8
Aug 10 14:37:29 dist-test-slave-h6n0 krb5kdc[22584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372649, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:29.636659 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:29.616843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:44009 (local address 127.21.55.126:37319)
0810 14:37:29.617376 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:29.617380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:29.618199 (+ 819us) server_negotiation.cc:407] Connection header received
0810 14:37:29.620283 (+ 2084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:29.620289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:29.620390 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:29.620538 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:29.622252 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.623324 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.625330 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.625793 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.625910 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:29.629327 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:29.629332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:29.629336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:29.629380 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:29.632173 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:29.633373 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:29.633390 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:29.633392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:29.633530 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:29.634131 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:29.634135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:29.634137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:29.634440 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:29.634654 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:29.635959 (+ 1305us) server_negotiation.cc:299] Negotiation successful
0810 14:37:29.636237 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":277,"threads_started":1}
I20260810 14:37:29.637953 23347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:29.617288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:37319 (local address 127.21.55.67:44009)
0810 14:37:29.618027 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:29.618079 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:29.619982 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:29.620757 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:29.620765 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:29.621435 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:29.622084 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:29.622102 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.623488 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.623494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:29.625158 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:29.625171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.626001 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.626006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:29.626200 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:29.626907 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:29.626928 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:29.629131 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:29.632468 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:29.632480 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:29.632499 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:29.633181 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:29.633721 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:29.633726 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:29.633729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:29.633986 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:29.634767 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:29.634775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:29.635508 (+ 733us) client_negotiation.cc:769] Sending connection context
0810 14:37:29.635884 (+ 376us) client_negotiation.cc:240] Negotiation successful
0810 14:37:29.636461 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":288,"threads_started":1}
I20260810 14:37:29.639369 23344 heartbeater.cc:347] Connected to a master server at 127.21.55.126:37319
I20260810 14:37:29.639657 23344 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:29.640372 23344 heartbeater.cc:511] Master 127.21.55.126:37319 requested a full tablet report, sending...
I20260810 14:37:29.642269 22646 ts_manager.cc:194] Registered new tserver with Master: b3a8327cd8e0459d85acafd0e21522af (127.21.55.67:43609)
I20260810 14:37:29.643186 22646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:44009
I20260810 14:37:29.650794 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:37:29.675298 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:29.658338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39110 (local address 127.21.55.126:37319)
0810 14:37:29.658518 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:29.658523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:29.658972 (+ 449us) server_negotiation.cc:407] Connection header received
0810 14:37:29.659115 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:29.659118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:29.659186 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:29.659276 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:29.660940 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.661813 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.663570 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.664056 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.664173 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:29.667810 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:29.667817 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:29.667821 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:29.667870 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:29.671781 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:29.672677 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:29.672683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:29.672685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:29.672999 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:29.673421 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:29.673425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:29.673427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:29.673781 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:29.674009 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:29.674834 (+ 825us) server_negotiation.cc:299] Negotiation successful
0810 14:37:29.675063 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:37:29.683398 22646 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:37:29.691756 22646 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:37:29.721141 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:29.707454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49876 (local address 127.21.55.67:43609)
0810 14:37:29.708581 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:29.708595 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:29.710076 (+ 1481us) server_negotiation.cc:407] Connection header received
0810 14:37:29.710601 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:29.710611 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:29.710922 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:29.711243 (+ 321us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:29.712701 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.714824 (+ 2123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.718049 (+ 3225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.719459 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.719607 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:29.720086 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:29.720453 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 14:37:29.720696 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":888,"thread_start_us":1194,"threads_started":1}
I20260810 14:37:29.721177 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:29.706859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:43609 (local address 127.0.0.1:49876)
0810 14:37:29.709876 (+ 3017us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:29.709935 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:29.710290 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:29.711728 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:29.711733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:29.711802 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:29.712457 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:29.712474 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.715062 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.715067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:29.717826 (+ 2759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:29.717843 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.719790 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.719802 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:29.720071 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:29.720146 (+ 75us) client_negotiation.cc:769] Sending connection context
0810 14:37:29.720371 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 14:37:29.720644 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2656,"mutex_wait_us":59,"thread_start_us":924,"threads_started":1}
I20260810 14:37:29.721884 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:29.708578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.21.55.65:45165)
0810 14:37:29.709497 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:29.709508 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:29.710097 (+ 589us) server_negotiation.cc:407] Connection header received
0810 14:37:29.710609 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:29.710618 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:29.710938 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:29.711232 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:29.712559 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.714402 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.717594 (+ 3192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.719433 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.719571 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:29.720024 (+ 453us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:29.720735 (+ 711us) server_negotiation.cc:299] Negotiation successful
0810 14:37:29.720966 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":553,"threads_started":1}
I20260810 14:37:29.722988 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:29.708034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45165 (local address 127.0.0.1:42428)
0810 14:37:29.709979 (+ 1945us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:29.710003 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:29.710290 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:29.711637 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:29.711645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:29.711695 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:29.712268 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:29.712285 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.714770 (+ 2485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.714779 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:29.717326 (+ 2547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:29.717346 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.720279 (+ 2933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.720288 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:29.720498 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:29.720529 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 14:37:29.722273 (+ 1744us) client_negotiation.cc:240] Negotiation successful
0810 14:37:29.722434 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1825,"mutex_wait_us":64,"spinlock_wait_cycles":50688,"thread_start_us":1636,"threads_started":1}
I20260810 14:37:29.723186 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:29.708850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:35803 (local address 127.0.0.1:56348)
0810 14:37:29.711518 (+ 2668us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:29.711547 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:29.711707 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:29.712778 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:29.712786 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:29.712820 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:29.713560 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:29.713575 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.716335 (+ 2760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.716341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:29.718967 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:29.718984 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.722274 (+ 3290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.722281 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:29.722459 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:29.722495 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 14:37:29.722629 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 14:37:29.722769 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2526,"mutex_wait_us":116,"spinlock_wait_cycles":10496,"thread_start_us":2377,"threads_started":1}
I20260810 14:37:29.723658 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:29.709517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56348 (local address 127.21.55.66:35803)
0810 14:37:29.710837 (+ 1320us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:29.710851 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:29.711687 (+ 836us) server_negotiation.cc:407] Connection header received
0810 14:37:29.711914 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:29.711925 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:29.712243 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:29.712497 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:29.713856 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.716090 (+ 2234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.719298 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:29.721681 (+ 2383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:29.721894 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:29.722105 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:29.723127 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0810 14:37:29.723344 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1112,"thread_start_us":935,"threads_started":1}
I20260810 14:37:29.726222 23239 tablet_service.cc:1467] Processing CreateTablet for tablet 46638d9e8a8847eda20ffb776da4ffbc (DEFAULT_TABLE table=test-table [id=9add2b0dbba743e38353bc56d6fd00f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:29.726433 22828 tablet_service.cc:1467] Processing CreateTablet for tablet 46638d9e8a8847eda20ffb776da4ffbc (DEFAULT_TABLE table=test-table [id=9add2b0dbba743e38353bc56d6fd00f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:29.727300 23037 tablet_service.cc:1467] Processing CreateTablet for tablet 46638d9e8a8847eda20ffb776da4ffbc (DEFAULT_TABLE table=test-table [id=9add2b0dbba743e38353bc56d6fd00f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:29.727862 22828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46638d9e8a8847eda20ffb776da4ffbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:29.727936 23239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46638d9e8a8847eda20ffb776da4ffbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:29.728644 23037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46638d9e8a8847eda20ffb776da4ffbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:29.750699 23380 tablet_bootstrap.cc:492] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb: Bootstrap starting.
I20260810 14:37:29.750854 23376 tablet_bootstrap.cc:492] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748: Bootstrap starting.
I20260810 14:37:29.756726 23377 tablet_bootstrap.cc:492] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af: Bootstrap starting.
I20260810 14:37:29.757411 23380 tablet_bootstrap.cc:654] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:29.758836 23380 log.cc:824] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:29.761019 23377 tablet_bootstrap.cc:654] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:29.762274 23376 tablet_bootstrap.cc:654] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:29.762581 23377 log.cc:824] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:29.763738 23376 log.cc:824] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:29.766089 23377 tablet_bootstrap.cc:492] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af: No bootstrap required, opened a new log
I20260810 14:37:29.766088 23380 tablet_bootstrap.cc:492] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb: No bootstrap required, opened a new log
I20260810 14:37:29.766395 23380 ts_tablet_manager.cc:1397] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.006s
I20260810 14:37:29.766433 23377 ts_tablet_manager.cc:1397] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260810 14:37:29.766911 23376 tablet_bootstrap.cc:492] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748: No bootstrap required, opened a new log
I20260810 14:37:29.767184 23376 ts_tablet_manager.cc:1397] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.000s
I20260810 14:37:29.774211 23377 raft_consensus.cc:348] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a8327cd8e0459d85acafd0e21522af" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43609 } } peers { permanent_uuid: "2cc65c7143a04891be5c8a27536e4748" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 35803 } } peers { permanent_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45165 } }
I20260810 14:37:29.774654 23377 raft_consensus.cc:374] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:29.774773 23377 raft_consensus.cc:729] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a8327cd8e0459d85acafd0e21522af, State: Initialized, Role: FOLLOWER
I20260810 14:37:29.775420 23377 consensus_queue.cc:260] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a8327cd8e0459d85acafd0e21522af" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43609 } } peers { permanent_uuid: "2cc65c7143a04891be5c8a27536e4748" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 35803 } } peers { permanent_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45165 } }
I20260810 14:37:29.776639 23344 heartbeater.cc:503] Master 127.21.55.126:37319 was elected leader, sending a full tablet report...
I20260810 14:37:29.777230 23380 raft_consensus.cc:348] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a8327cd8e0459d85acafd0e21522af" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43609 } } peers { permanent_uuid: "2cc65c7143a04891be5c8a27536e4748" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 35803 } } peers { permanent_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45165 } }
I20260810 14:37:29.777666 23380 raft_consensus.cc:374] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:29.777558 23377 ts_tablet_manager.cc:1428] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260810 14:37:29.777768 23380 raft_consensus.cc:729] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f0ed51f3ba9416b9f2c78ca56b498fb, State: Initialized, Role: FOLLOWER
I20260810 14:37:29.778400 23380 consensus_queue.cc:260] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a8327cd8e0459d85acafd0e21522af" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43609 } } peers { permanent_uuid: "2cc65c7143a04891be5c8a27536e4748" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 35803 } } peers { permanent_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45165 } }
I20260810 14:37:29.779512 23376 raft_consensus.cc:348] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a8327cd8e0459d85acafd0e21522af" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43609 } } peers { permanent_uuid: "2cc65c7143a04891be5c8a27536e4748" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 35803 } } peers { permanent_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45165 } }
I20260810 14:37:29.780011 23376 raft_consensus.cc:374] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:29.780128 23376 raft_consensus.cc:729] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc65c7143a04891be5c8a27536e4748, State: Initialized, Role: FOLLOWER
I20260810 14:37:29.780714 23376 consensus_queue.cc:260] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a8327cd8e0459d85acafd0e21522af" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43609 } } peers { permanent_uuid: "2cc65c7143a04891be5c8a27536e4748" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 35803 } } peers { permanent_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45165 } }
I20260810 14:37:29.781952 23134 heartbeater.cc:503] Master 127.21.55.126:37319 was elected leader, sending a full tablet report...
I20260810 14:37:29.782752 23376 ts_tablet_manager.cc:1428] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260810 14:37:29.784652 23380 ts_tablet_manager.cc:1428] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb: Time spent starting tablet: real 0.018s user 0.008s sys 0.008s
W20260810 14:37:29.801518 23135 tablet.cc:2369] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:37:29.865840 23345 tablet.cc:2369] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:37:29.937762 22934 tablet.cc:2369] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:37:30.048156 23386 raft_consensus.cc:482] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:37:30.048478 23386 raft_consensus.cc:504] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a8327cd8e0459d85acafd0e21522af" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43609 } } peers { permanent_uuid: "2cc65c7143a04891be5c8a27536e4748" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 35803 } } peers { permanent_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45165 } }
I20260810 14:37:30.051225 23386 leader_election.cc:288] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cc65c7143a04891be5c8a27536e4748 (127.21.55.66:35803), 6f0ed51f3ba9416b9f2c78ca56b498fb (127.21.55.65:45165)
I20260810 14:37:30.058176 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:30.051779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:53477 (local address 127.21.55.66:35803)
0810 14:37:30.051991 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:30.051998 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:30.052264 (+ 266us) server_negotiation.cc:407] Connection header received
0810 14:37:30.052405 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:30.052413 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:30.052508 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:30.052637 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:30.053303 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.054427 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.056362 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.057223 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.057407 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:30.057482 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:30.057787 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 14:37:30.057922 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:37:30.058185 23347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:30.051705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:35803 (local address 127.21.55.67:53477)
0810 14:37:30.052083 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:30.052104 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:30.052245 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:30.052830 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:30.052835 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:30.052860 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:30.053127 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:30.053138 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.054591 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.054597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:30.056185 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:30.056197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.057388 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.057394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:30.057614 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:30.057648 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 14:37:30.057743 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 14:37:30.057878 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":4,"spinlock_wait_cycles":27008}
I20260810 14:37:30.059868 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:30.051843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:35415 (local address 127.21.55.65:45165)
0810 14:37:30.052192 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:30.052198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:30.052847 (+ 649us) server_negotiation.cc:407] Connection header received
0810 14:37:30.053233 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:30.053240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:30.053326 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:30.053437 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:30.054024 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.055189 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.057584 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.058866 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.059047 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:30.059156 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:30.059473 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 14:37:30.059623 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198}
I20260810 14:37:30.059640 23067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46638d9e8a8847eda20ffb776da4ffbc" candidate_uuid: "b3a8327cd8e0459d85acafd0e21522af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc65c7143a04891be5c8a27536e4748" is_pre_election: true
I20260810 14:37:30.059908 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:30.052137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45165 (local address 127.21.55.67:35415)
0810 14:37:30.052750 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:30.052767 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:30.053006 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:30.053463 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:30.053474 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:30.053499 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:30.053844 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:30.053855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.055351 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.055357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:30.057446 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:30.057461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.059074 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.059081 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:30.059248 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:30.059291 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 14:37:30.059411 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 14:37:30.059554 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":3,"thread_start_us":581,"threads_started":1}
I20260810 14:37:30.060554 23067 raft_consensus.cc:2454] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3a8327cd8e0459d85acafd0e21522af in term 0.
I20260810 14:37:30.061120 22863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46638d9e8a8847eda20ffb776da4ffbc" candidate_uuid: "b3a8327cd8e0459d85acafd0e21522af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb" is_pre_election: true
I20260810 14:37:30.061455 23165 leader_election.cc:302] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc65c7143a04891be5c8a27536e4748, b3a8327cd8e0459d85acafd0e21522af; no voters:
I20260810 14:37:30.061653 22863 raft_consensus.cc:2454] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3a8327cd8e0459d85acafd0e21522af in term 0.
I20260810 14:37:30.061863 23386 raft_consensus.cc:2781] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:37:30.061988 23386 raft_consensus.cc:482] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:37:30.062124 23386 raft_consensus.cc:3037] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:30.064730 23386 raft_consensus.cc:504] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a8327cd8e0459d85acafd0e21522af" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43609 } } peers { permanent_uuid: "2cc65c7143a04891be5c8a27536e4748" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 35803 } } peers { permanent_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45165 } }
I20260810 14:37:30.065404 23386 leader_election.cc:288] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [CANDIDATE]: Term 1 election: Requested vote from peers 2cc65c7143a04891be5c8a27536e4748 (127.21.55.66:35803), 6f0ed51f3ba9416b9f2c78ca56b498fb (127.21.55.65:45165)
I20260810 14:37:30.065799 23067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46638d9e8a8847eda20ffb776da4ffbc" candidate_uuid: "b3a8327cd8e0459d85acafd0e21522af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cc65c7143a04891be5c8a27536e4748"
I20260810 14:37:30.065901 22863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46638d9e8a8847eda20ffb776da4ffbc" candidate_uuid: "b3a8327cd8e0459d85acafd0e21522af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb"
I20260810 14:37:30.066008 23067 raft_consensus.cc:3037] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:30.066150 22863 raft_consensus.cc:3037] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:30.068316 23067 raft_consensus.cc:2454] T 46638d9e8a8847eda20ffb776da4ffbc P 2cc65c7143a04891be5c8a27536e4748 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3a8327cd8e0459d85acafd0e21522af in term 1.
I20260810 14:37:30.068439 22863 raft_consensus.cc:2454] T 46638d9e8a8847eda20ffb776da4ffbc P 6f0ed51f3ba9416b9f2c78ca56b498fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3a8327cd8e0459d85acafd0e21522af in term 1.
I20260810 14:37:30.068717 23165 leader_election.cc:302] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cc65c7143a04891be5c8a27536e4748, b3a8327cd8e0459d85acafd0e21522af; no voters:
I20260810 14:37:30.069069 23386 raft_consensus.cc:2781] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:30.069505 23386 raft_consensus.cc:686] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [term 1 LEADER]: Becoming Leader. State: Replica: b3a8327cd8e0459d85acafd0e21522af, State: Running, Role: LEADER
I20260810 14:37:30.070139 23386 consensus_queue.cc:237] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a8327cd8e0459d85acafd0e21522af" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43609 } } peers { permanent_uuid: "2cc65c7143a04891be5c8a27536e4748" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 35803 } } peers { permanent_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45165 } }
I20260810 14:37:30.076094 22643 catalog_manager.cc:5495] T 46638d9e8a8847eda20ffb776da4ffbc P b3a8327cd8e0459d85acafd0e21522af reported cstate change: term changed from 0 to 1, leader changed from <none> to b3a8327cd8e0459d85acafd0e21522af (127.21.55.67). New cstate: current_term: 1 leader_uuid: "b3a8327cd8e0459d85acafd0e21522af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a8327cd8e0459d85acafd0e21522af" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cc65c7143a04891be5c8a27536e4748" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 35803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f0ed51f3ba9416b9f2c78ca56b498fb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45165 } health_report { overall_health: UNKNOWN } } }
Aug 10 14:37:30 dist-test-slave-h6n0 krb5kdc[22584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:37:30.104029 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:30.087580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.21.55.65:45165)
0810 14:37:30.087828 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:30.087851 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:30.088587 (+ 736us) server_negotiation.cc:407] Connection header received
0810 14:37:30.088666 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:30.088671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:30.088737 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:30.088839 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:30.090516 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.091384 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.093026 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.093483 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.093599 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:30.098094 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:30.098109 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:30.098126 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:30.098158 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:30.101300 (+ 3142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.102017 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.102024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.102027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.102171 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.102674 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.102677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.102679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.103048 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:30.103277 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:30.103579 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 14:37:30.103740 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
Aug 10 14:37:30 dist-test-slave-h6n0 krb5kdc[22584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:37:30 dist-test-slave-h6n0 krb5kdc[22584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372650, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:37:30.157438 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:30.141586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42456 (local address 127.21.55.65:45165)
0810 14:37:30.141814 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:30.141824 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:30.142204 (+ 380us) server_negotiation.cc:407] Connection header received
0810 14:37:30.142424 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:30.142435 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:30.142500 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:30.142660 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:30.144076 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.144825 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.146192 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.146791 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.146929 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:30.151448 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:30.151454 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:30.151457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:30.151523 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:30.154645 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.155380 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.155387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.155390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.155549 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.156048 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.156053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.156056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.156458 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:30.156753 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:30.156925 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 14:37:30.157100 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Aug 10 14:37:30 dist-test-slave-h6n0 krb5kdc[22584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:37:30 dist-test-slave-h6n0 krb5kdc[22584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:37:30.215838 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:30.198620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42466 (local address 127.21.55.65:45165)
0810 14:37:30.198855 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:30.198864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:30.199052 (+ 188us) server_negotiation.cc:407] Connection header received
0810 14:37:30.199164 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:30.199168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:30.199235 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:30.199642 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:30.201011 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.201738 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.203912 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.204372 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.204479 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:30.209065 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:30.209070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:30.209074 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:30.209106 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:30.212528 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.213457 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.213461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.213464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.213611 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.214037 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.214042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.214044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.214420 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:30.214805 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:30.215104 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 14:37:30.215414 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 14:37:30.219137 22801 tablet_service.cc:2908] Rejecting scan request for tablet 46638d9e8a8847eda20ffb776da4ffbc: Uninitialized: clean time has not yet been initialized
I20260810 14:37:30.223801 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22736
I20260810 14:37:30.241289 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:30.225466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42476 (local address 127.21.55.65:45165)
0810 14:37:30.225774 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:30.225785 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:30.226461 (+ 676us) server_negotiation.cc:407] Connection header received
0810 14:37:30.226701 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:30.226705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:30.226801 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:30.226898 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:30.228578 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.229499 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.230962 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.231446 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.231571 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:30.234283 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:30.234289 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:30.234292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:30.234326 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:30.237280 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.238292 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.238299 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.238302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.238502 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.239471 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.239476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.239478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.239800 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:30.240063 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:30.240540 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 14:37:30.240828 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260810 14:37:30.242408 22928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:30.346117 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 22736
I20260810 14:37:30.372272 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22943
Aug 10 14:37:30 dist-test-slave-h6n0 krb5kdc[22584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:37:30.389245 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:30.373136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56352 (local address 127.21.55.66:35803)
0810 14:37:30.373316 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:30.373321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:30.373339 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:37:30.373445 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:30.373449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:30.373518 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:30.373604 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:30.375051 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.376024 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.377312 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.377718 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.377833 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:30.382458 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:30.382481 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:30.382497 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:30.382808 (+ 311us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:30.385820 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.386813 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.386827 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.386831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.387112 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.387755 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.387760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.387763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.388196 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:30.388459 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:30.388699 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 14:37:30.388864 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:37:30.390231 23127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:30.492214 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 22943
I20260810 14:37:30.526696 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23143
Aug 10 14:37:30 dist-test-slave-h6n0 krb5kdc[22584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:37:30.545116 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:30.527763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49888 (local address 127.21.55.67:43609)
0810 14:37:30.528306 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:30.528311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:30.528328 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:37:30.528384 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:30.528388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:30.528448 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:30.528611 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:30.529940 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.530704 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.532198 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:30.532640 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:30.532797 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:30.537422 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:30.537448 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:30.537471 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:30.537525 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:30.541483 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.542803 (+ 1320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.542813 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.542815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.543034 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:30.543589 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:30.543593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:30.543595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:30.543994 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:30.544266 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:30.544580 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 14:37:30.544805 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":300,"threads_started":1}
I20260810 14:37:30.546062 23337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:30.644358 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23143
I20260810 14:37:30.671532 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22601
I20260810 14:37:30.673576 22687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:30.795809 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 22601
2026-08-10T14:37:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5120 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:37:30 dist-test-slave-h6n0 krb5kdc[23419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:37:30 dist-test-slave-h6n0 krb5kdc[23419](info): set up 2 sockets
Aug 10 14:37:30 dist-test-slave-h6n0 krb5kdc[23419](info): commencing operation
krb5kdc: starting...
W20260810 14:37:32.200450 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:37:32 dist-test-slave-h6n0 krb5kdc[23419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:37:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:37:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:37:32.440001 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:46841
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:34313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:46841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:32.827979 23437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:32.828337 23437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:32.828537 23437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:32.836795 23437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:37:32.836917 23437 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 14:37:32.836990 23437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:32.837087 23437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:32.837174 23437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:37:32.837273 23437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:37:32.847505 23437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:46841
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:46841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:32.848912 23437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:32.850374 23437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:32.864586 23447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:32.864610 23445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:32.865118 23444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:32.865932 23437 server_base.cc:1034] running on GCE node
I20260810 14:37:32.867093 23437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:32.868566 23437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:32.869865 23437 hybrid_clock.cc:648] HybridClock initialized: now 1786372652869809 us; error 58 us; skew 500 ppm
I20260810 14:37:32.870244 23437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:32 dist-test-slave-h6n0 krb5kdc[23419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372652, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:32.879554 23437 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:37:32.882072 23437 webserver.cc:460] Webserver started at http://127.21.55.126:35281/ using document root <none> and password file <none>
I20260810 14:37:32.883221 23437 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:32.883404 23437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:32.883844 23437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:32.888144 23437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "af26d05611f74455b859c52c01d2a900"
format_stamp: "Formatted at 2026-08-10 14:37:32 on dist-test-slave-h6n0"
server_key: "1f6d8f77c8fb4dd9fa92cb9c1134585d"
server_key_iv: "60db9c847bb6f46d9e9c2244a6fecc5d"
server_key_version: "encryptionkey@0"
I20260810 14:37:32.889163 23437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "af26d05611f74455b859c52c01d2a900"
format_stamp: "Formatted at 2026-08-10 14:37:32 on dist-test-slave-h6n0"
server_key: "1f6d8f77c8fb4dd9fa92cb9c1134585d"
server_key_iv: "60db9c847bb6f46d9e9c2244a6fecc5d"
server_key_version: "encryptionkey@0"
I20260810 14:37:32.896720 23437 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260810 14:37:32.901822 23460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:32.903584 23437 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 14:37:32.903807 23437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "af26d05611f74455b859c52c01d2a900"
format_stamp: "Formatted at 2026-08-10 14:37:32 on dist-test-slave-h6n0"
server_key: "1f6d8f77c8fb4dd9fa92cb9c1134585d"
server_key_iv: "60db9c847bb6f46d9e9c2244a6fecc5d"
server_key_version: "encryptionkey@0"
I20260810 14:37:32.904069 23437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:32.933249 23437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:32.937801 23437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:32.938102 23437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:32.963794 23437 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:46841
I20260810 14:37:32.963922 23553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:46841 every 8 connection(s)
I20260810 14:37:32.965359 23437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:37:32.969431 23555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:32.974502 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23437
I20260810 14:37:32.974815 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:37:32.975740 21725 external_mini_cluster.cc:1428] Setting key 3547a55de2d167f3d0b8e1b63b1e7277
I20260810 14:37:32.982769 23555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900: Bootstrap starting.
I20260810 14:37:32.986094 23555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:32.987350 23555 log.cc:824] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:32.990350 23555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900: No bootstrap required, opened a new log
Aug 10 14:37:32 dist-test-slave-h6n0 krb5kdc[23419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:32.999032 23555 raft_consensus.cc:348] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af26d05611f74455b859c52c01d2a900" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46841 } }
I20260810 14:37:32.999411 23555 raft_consensus.cc:374] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:32.999547 23555 raft_consensus.cc:729] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af26d05611f74455b859c52c01d2a900, State: Initialized, Role: FOLLOWER
I20260810 14:37:33.000164 23555 consensus_queue.cc:260] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af26d05611f74455b859c52c01d2a900" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46841 } }
I20260810 14:37:33.000427 23555 raft_consensus.cc:388] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:37:33.000581 23555 raft_consensus.cc:482] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:37:33.000790 23555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:33.002337 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:32.978105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48156 (local address 127.21.55.126:46841)
0810 14:37:32.979435 (+ 1330us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:32.979453 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:32.979518 (+ 65us) server_negotiation.cc:407] Connection header received
0810 14:37:32.981790 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:32.981823 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:32.982370 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:32.982984 (+ 614us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:32.985244 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:32.987314 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:32.989191 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:32.989889 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:32.990093 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:32.994668 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:32.994677 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:32.994697 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:32.994735 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:32.997873 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:32.998620 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:32.998629 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:32.998647 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:32.998813 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:32.999205 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:32.999211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:32.999214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:32.999967 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:33.000249 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:33.000794 (+ 545us) server_negotiation.cc:299] Negotiation successful
0810 14:37:33.001320 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973,"thread_start_us":658,"threads_started":1}
I20260810 14:37:33.003515 23555 raft_consensus.cc:504] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af26d05611f74455b859c52c01d2a900" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46841 } }
I20260810 14:37:33.004144 23555 leader_election.cc:302] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af26d05611f74455b859c52c01d2a900; no voters:
I20260810 14:37:33.004865 23555 leader_election.cc:288] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:37:33.005128 23565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:33.006444 23565 raft_consensus.cc:686] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [term 1 LEADER]: Becoming Leader. State: Replica: af26d05611f74455b859c52c01d2a900, State: Running, Role: LEADER
I20260810 14:37:33.007259 23555 sys_catalog.cc:564] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:37:33.007143 23565 consensus_queue.cc:237] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af26d05611f74455b859c52c01d2a900" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46841 } }
I20260810 14:37:33.015653 23568 sys_catalog.cc:455] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af26d05611f74455b859c52c01d2a900. Latest consensus state: current_term: 1 leader_uuid: "af26d05611f74455b859c52c01d2a900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af26d05611f74455b859c52c01d2a900" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46841 } } }
I20260810 14:37:33.016117 23568 sys_catalog.cc:458] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:33.016956 23566 sys_catalog.cc:455] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af26d05611f74455b859c52c01d2a900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af26d05611f74455b859c52c01d2a900" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46841 } } }
I20260810 14:37:33.017280 23566 sys_catalog.cc:458] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900 [sys.catalog]: This master's current role is: LEADER
W20260810 14:37:33.019954 23583 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:37:33.020073 23583 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:37:33.020213 23585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:37:33.021633 23585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:37:33.034142 23585 catalog_manager.cc:1347] Generated new cluster ID: f855a1766e36411abb2c4dd53118a628
I20260810 14:37:33.034296 23585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:37:33.060267 23585 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:37:33.062130 23585 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:37:33.069636 23585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900: Generated new TSK 0
I20260810 14:37:33.070569 23585 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:37:33.142258 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46841
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:34313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:33.511950 23592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:33.512285 23592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:33.512444 23592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:33.523262 23592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:33.523401 23592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:33.523501 23592 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:37:33.523605 23592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:37:33.537499 23592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.55.126:46841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:33.539548 23592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:33.541052 23592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:33.556864 23598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:33.557018 23599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:33.557056 23604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:33.558393 23592 server_base.cc:1034] running on GCE node
I20260810 14:37:33.559438 23592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:33.560985 23592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:33.562284 23592 hybrid_clock.cc:648] HybridClock initialized: now 1786372653562207 us; error 47 us; skew 500 ppm
I20260810 14:37:33.562731 23592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:33 dist-test-slave-h6n0 krb5kdc[23419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372653, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:33.567435 23592 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:37:33.570065 23592 webserver.cc:460] Webserver started at http://127.21.55.65:44445/ using document root <none> and password file <none>
I20260810 14:37:33.571319 23592 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:33.571534 23592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:33.572098 23592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:33.575968 23592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "c6f95c371d9a4f9990154f0c9535202b"
format_stamp: "Formatted at 2026-08-10 14:37:33 on dist-test-slave-h6n0"
server_key: "c9d6a26126d5a9ad6815a8454c961567"
server_key_iv: "90b46229c023a5d663c373b5720f849d"
server_key_version: "encryptionkey@0"
I20260810 14:37:33.576871 23592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "c6f95c371d9a4f9990154f0c9535202b"
format_stamp: "Formatted at 2026-08-10 14:37:33 on dist-test-slave-h6n0"
server_key: "c9d6a26126d5a9ad6815a8454c961567"
server_key_iv: "90b46229c023a5d663c373b5720f849d"
server_key_version: "encryptionkey@0"
I20260810 14:37:33.584431 23592 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.005s
I20260810 14:37:33.588626 23615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:33.590245 23592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:37:33.590485 23592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "c6f95c371d9a4f9990154f0c9535202b"
format_stamp: "Formatted at 2026-08-10 14:37:33 on dist-test-slave-h6n0"
server_key: "c9d6a26126d5a9ad6815a8454c961567"
server_key_iv: "90b46229c023a5d663c373b5720f849d"
server_key_version: "encryptionkey@0"
I20260810 14:37:33.591085 23592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:33.613659 23592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:33.619527 23592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:33.619838 23592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:33.623080 23592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:33.623195 23592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:33.623354 23592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:33.623430 23592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:37:33 dist-test-slave-h6n0 krb5kdc[23419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372653, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:33.652957 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:33.623794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:52115 (local address 127.21.55.126:46841)
0810 14:37:33.624449 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:33.624454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:33.625659 (+ 1205us) server_negotiation.cc:407] Connection header received
0810 14:37:33.628123 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:33.628129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:33.628237 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:33.628361 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:33.630135 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:33.631766 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:33.635391 (+ 3625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:33.636285 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:33.636480 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:33.643280 (+ 6800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:33.643288 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:33.643294 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:33.643348 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:33.648091 (+ 4743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:33.649474 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:33.649491 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:33.649497 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:33.649754 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:33.650420 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:33.650428 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:33.650434 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:33.650890 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:33.651075 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:33.652230 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0810 14:37:33.652508 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":444,"threads_started":1}
I20260810 14:37:33.654912 23628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:33.624512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46841 (local address 127.21.55.65:52115)
0810 14:37:33.625487 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:33.625531 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:33.627800 (+ 2269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:33.628616 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:33.628624 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:33.629203 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:33.629838 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:33.629856 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:33.632028 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:33.632038 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:33.635152 (+ 3114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:33.635181 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:33.636626 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:33.636639 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:33.637055 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:33.638189 (+ 1134us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:33.638234 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:33.642785 (+ 4551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:33.648305 (+ 5520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:33.648320 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:33.648346 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:33.649286 (+ 940us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:33.649900 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:33.649909 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:33.649914 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:33.650187 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:33.651058 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:33.651068 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:33.651713 (+ 645us) client_negotiation.cc:769] Sending connection context
0810 14:37:33.652101 (+ 388us) client_negotiation.cc:240] Negotiation successful
0810 14:37:33.652796 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":508,"threads_started":1}
I20260810 14:37:33.675779 23592 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:37417
I20260810 14:37:33.675828 23787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:37417 every 8 connection(s)
I20260810 14:37:33.677316 23592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:37:33.678676 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23592
I20260810 14:37:33.678925 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:37:33.679780 21725 external_mini_cluster.cc:1428] Setting key e3fc884b0cff8387423f826f66bc3f4d
I20260810 14:37:33.681010 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:33.662904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46841 (local address 127.21.55.65:51723)
0810 14:37:33.663875 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:33.663902 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:33.664074 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:33.664765 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:33.664771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:33.665613 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:33.666212 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:33.666235 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:33.668596 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:33.668605 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:33.671020 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:33.671035 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:33.672020 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:33.672027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:33.672223 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:33.673259 (+ 1036us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:33.673277 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:33.674158 (+ 881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:33.677538 (+ 3380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:33.677542 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:33.677545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:33.678025 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:33.678672 (+ 647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:33.678677 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:33.678680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:33.678857 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:33.679689 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:33.679694 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:33.679928 (+ 234us) client_negotiation.cc:769] Sending connection context
0810 14:37:33.680109 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 14:37:33.680492 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":607,"threads_started":1}
I20260810 14:37:33.682057 23789 heartbeater.cc:347] Connected to a master server at 127.21.55.126:46841
I20260810 14:37:33.682124 23627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:33.662448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:51723 (local address 127.21.55.126:46841)
0810 14:37:33.662663 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:33.662667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:33.664198 (+ 1531us) server_negotiation.cc:407] Connection header received
0810 14:37:33.664334 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:33.664341 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:33.664481 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:33.664679 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:33.666462 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:33.668228 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:33.671233 (+ 3005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:33.671702 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:33.671986 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:33.674392 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:33.674397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:33.674401 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:33.674435 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:33.677334 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:33.678285 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:33.678290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:33.678293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:33.678470 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:33.679038 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:33.679043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:33.679046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:33.679442 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:33.679605 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:33.681309 (+ 1704us) server_negotiation.cc:299] Negotiation successful
0810 14:37:33.681691 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 14:37:33.682623 23789 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:33.683579 23789 heartbeater.cc:511] Master 127.21.55.126:46841 requested a full tablet report, sending...
I20260810 14:37:33.688746 23487 ts_manager.cc:194] Registered new tserver with Master: c6f95c371d9a4f9990154f0c9535202b (127.21.55.65:37417)
I20260810 14:37:33.690433 23487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:52115
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:37:33.734478 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46841
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:34313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:34.126379 23794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:34.126957 23794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:34.127185 23794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:34.136798 23794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:34.136938 23794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:34.137058 23794 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:37:34.137151 23794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:37:34.148231 23794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.55.126:46841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:34.149574 23794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:34.151151 23794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:34.167339 23802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:34.167429 23808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:34.167384 23803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:34.168859 23794 server_base.cc:1034] running on GCE node
I20260810 14:37:34.169914 23794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:34.171073 23794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:34.172482 23794 hybrid_clock.cc:648] HybridClock initialized: now 1786372654172364 us; error 45 us; skew 500 ppm
I20260810 14:37:34.173100 23794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:34 dist-test-slave-h6n0 krb5kdc[23419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:34.177891 23794 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:37:34.179773 23794 webserver.cc:460] Webserver started at http://127.21.55.66:42797/ using document root <none> and password file <none>
I20260810 14:37:34.180567 23794 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:34.180716 23794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:34.181077 23794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:34.184481 23794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "9a901aa5bf1248c5972e6c1a7638700d"
format_stamp: "Formatted at 2026-08-10 14:37:34 on dist-test-slave-h6n0"
server_key: "5fe6d1f932beb7aa336ac07f9b0fa5aa"
server_key_iv: "da101b058007769dd61466f3cdcef01d"
server_key_version: "encryptionkey@0"
I20260810 14:37:34.185335 23794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "9a901aa5bf1248c5972e6c1a7638700d"
format_stamp: "Formatted at 2026-08-10 14:37:34 on dist-test-slave-h6n0"
server_key: "5fe6d1f932beb7aa336ac07f9b0fa5aa"
server_key_iv: "da101b058007769dd61466f3cdcef01d"
server_key_version: "encryptionkey@0"
I20260810 14:37:34.190902 23794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 14:37:34.195062 23819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:34.196557 23794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:37:34.196761 23794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "9a901aa5bf1248c5972e6c1a7638700d"
format_stamp: "Formatted at 2026-08-10 14:37:34 on dist-test-slave-h6n0"
server_key: "5fe6d1f932beb7aa336ac07f9b0fa5aa"
server_key_iv: "da101b058007769dd61466f3cdcef01d"
server_key_version: "encryptionkey@0"
I20260810 14:37:34.197124 23794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:34.210304 23794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:34.215302 23794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:34.215655 23794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:34.218984 23794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:34.219116 23794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:34.219321 23794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:34.219414 23794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:37:34 dist-test-slave-h6n0 krb5kdc[23419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:34.249191 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.220375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:41893 (local address 127.21.55.126:46841)
0810 14:37:34.221016 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:34.221025 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:34.222942 (+ 1917us) server_negotiation.cc:407] Connection header received
0810 14:37:34.226296 (+ 3354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:34.226305 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:34.226426 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:34.226600 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:34.229666 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.230913 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.232741 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.233218 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.233366 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:34.240243 (+ 6877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:34.240253 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:34.240260 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:34.240316 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:34.244927 (+ 4611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:34.245935 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:34.245944 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:34.245950 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:34.246153 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:34.246713 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:34.246718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:34.246720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:34.247108 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:34.247302 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:34.248518 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0810 14:37:34.248718 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":324,"threads_started":1}
I20260810 14:37:34.251487 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.221276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46841 (local address 127.21.55.66:41893)
0810 14:37:34.222692 (+ 1416us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:34.222768 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:34.225831 (+ 3063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:34.226926 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:34.226941 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:34.228128 (+ 1187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:34.229383 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:34.229420 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.231086 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.231092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:34.232578 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:34.232592 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.233492 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.233501 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:34.233746 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:34.234687 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:34.234713 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:34.239852 (+ 5139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:34.245139 (+ 5287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:34.245151 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:34.245173 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:34.245781 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:34.246293 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:34.246296 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:34.246298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:34.246508 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:34.247286 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:34.247302 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:34.247937 (+ 635us) client_negotiation.cc:769] Sending connection context
0810 14:37:34.248347 (+ 410us) client_negotiation.cc:240] Negotiation successful
0810 14:37:34.249134 (+ 787us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"thread_start_us":779,"threads_started":1}
I20260810 14:37:34.282879 23794 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:46853
I20260810 14:37:34.282931 23983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:46853 every 8 connection(s)
I20260810 14:37:34.283457 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.262359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:41235 (local address 127.21.55.126:46841)
0810 14:37:34.262612 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:34.262617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:34.263212 (+ 595us) server_negotiation.cc:407] Connection header received
0810 14:37:34.263421 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:34.263428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:34.263532 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:34.263643 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:34.265556 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.266754 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.269350 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.269832 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.269975 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:34.273222 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:34.273233 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:34.273240 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:34.273317 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:34.279121 (+ 5804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:34.280798 (+ 1677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:34.280804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:34.280807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:34.281004 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:34.281671 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:34.281678 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:34.281713 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:34.282137 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:34.282329 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:34.282957 (+ 628us) server_negotiation.cc:299] Negotiation successful
0810 14:37:34.283229 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:37:34.283730 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.262321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46841 (local address 127.21.55.66:41235)
0810 14:37:34.263061 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:34.263094 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:34.263279 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:34.263674 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:34.263679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:34.264573 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:34.265311 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:34.265331 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.266945 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.266950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:34.269136 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:34.269153 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.270053 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.270059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:34.270229 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:34.271359 (+ 1130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:34.271392 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:34.272893 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:34.279972 (+ 7079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:34.279978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:34.279982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:34.280598 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:34.281297 (+ 699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:34.281306 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:34.281312 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:34.281515 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:34.282302 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:34.282307 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:34.282586 (+ 279us) client_negotiation.cc:769] Sending connection context
0810 14:37:34.282817 (+ 231us) client_negotiation.cc:240] Negotiation successful
0810 14:37:34.283199 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":944,"threads_started":1}
I20260810 14:37:34.284777 23794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:37:34.287578 23984 heartbeater.cc:347] Connected to a master server at 127.21.55.126:46841
I20260810 14:37:34.287892 23984 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:34.288882 23984 heartbeater.cc:511] Master 127.21.55.126:46841 requested a full tablet report, sending...
I20260810 14:37:34.290685 23487 ts_manager.cc:194] Registered new tserver with Master: 9a901aa5bf1248c5972e6c1a7638700d (127.21.55.66:46853)
I20260810 14:37:34.291605 23487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:41893
I20260810 14:37:34.292033 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23794
I20260810 14:37:34.292313 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:37:34.292814 21725 external_mini_cluster.cc:1428] Setting key 75ccfbd318949d801940ea55b1258f80
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:37:34.346586 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46841
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:34313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:37:34.693753 23789 heartbeater.cc:503] Master 127.21.55.126:46841 was elected leader, sending a full tablet report...
W20260810 14:37:34.728304 23989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:34.728590 23989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:34.728690 23989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:34.736728 23989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:34.736924 23989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:34.737042 23989 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:37:34.737162 23989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:37:34.748944 23989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.55.126:46841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:34.750300 23989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:34.751653 23989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:34.767042 23996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:34.767091 24001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:34.767145 23998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:34.772220 23989 server_base.cc:1034] running on GCE node
I20260810 14:37:34.773054 23989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:34.774034 23989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:34.775315 23989 hybrid_clock.cc:648] HybridClock initialized: now 1786372654775234 us; error 59 us; skew 500 ppm
I20260810 14:37:34.775699 23989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:34 dist-test-slave-h6n0 krb5kdc[23419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:34.779484 23989 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:37:34.781250 23989 webserver.cc:460] Webserver started at http://127.21.55.67:33987/ using document root <none> and password file <none>
I20260810 14:37:34.782056 23989 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:34.782209 23989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:34.782645 23989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:34.786052 23989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "fd42b6727a734237afa13f05656e9591"
format_stamp: "Formatted at 2026-08-10 14:37:34 on dist-test-slave-h6n0"
server_key: "0dc8626b9a70cf5f888eca743a3d5bd7"
server_key_iv: "1d4b1de27e0144eafc655bfe0a4bfe96"
server_key_version: "encryptionkey@0"
I20260810 14:37:34.787001 23989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "fd42b6727a734237afa13f05656e9591"
format_stamp: "Formatted at 2026-08-10 14:37:34 on dist-test-slave-h6n0"
server_key: "0dc8626b9a70cf5f888eca743a3d5bd7"
server_key_iv: "1d4b1de27e0144eafc655bfe0a4bfe96"
server_key_version: "encryptionkey@0"
I20260810 14:37:34.793099 23989 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260810 14:37:34.797821 24015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:34.799873 23989 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260810 14:37:34.800112 23989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "fd42b6727a734237afa13f05656e9591"
format_stamp: "Formatted at 2026-08-10 14:37:34 on dist-test-slave-h6n0"
server_key: "0dc8626b9a70cf5f888eca743a3d5bd7"
server_key_iv: "1d4b1de27e0144eafc655bfe0a4bfe96"
server_key_version: "encryptionkey@0"
I20260810 14:37:34.800400 23989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:34.819315 23989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:34.825044 23989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:34.825362 23989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:34.828554 23989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:34.828673 23989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:34.828856 23989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:34.828943 23989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:37:34 dist-test-slave-h6n0 krb5kdc[23419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:34.855365 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.829085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:52035 (local address 127.21.55.126:46841)
0810 14:37:34.829926 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:34.829935 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:34.830994 (+ 1059us) server_negotiation.cc:407] Connection header received
0810 14:37:34.833077 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:34.833084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:34.833206 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:34.833375 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:34.835905 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.837061 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.839812 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.840246 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.840383 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:34.847617 (+ 7234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:34.847623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:34.847626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:34.847662 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:34.850736 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:34.852024 (+ 1288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:34.852031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:34.852036 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:34.852206 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:34.852856 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:34.852864 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:34.852868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:34.853165 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:34.853293 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:34.854499 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0810 14:37:34.854856 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":582,"threads_started":1}
I20260810 14:37:34.857441 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.829831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46841 (local address 127.21.55.67:52035)
0810 14:37:34.830803 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:34.830858 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:34.832781 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:34.833700 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:34.833711 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:34.834613 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:34.835596 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:34.835630 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.837242 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.837253 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:34.839621 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:34.839635 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.840592 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.840606 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:34.841018 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:34.841957 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:34.841995 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:34.847346 (+ 5351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:34.850936 (+ 3590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:34.850961 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:34.850984 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:34.851856 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:34.852358 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:34.852362 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:34.852364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:34.852686 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:34.853318 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:34.853330 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:34.853988 (+ 658us) client_negotiation.cc:769] Sending connection context
0810 14:37:34.854381 (+ 393us) client_negotiation.cc:240] Negotiation successful
0810 14:37:34.855130 (+ 749us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":436,"threads_started":1}
I20260810 14:37:34.883122 23989 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:44377
I20260810 14:37:34.883363 24193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:44377 every 8 connection(s)
I20260810 14:37:34.885099 23989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:37:34.887465 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.865962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:54525 (local address 127.21.55.126:46841)
0810 14:37:34.866237 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:34.866246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:34.866795 (+ 549us) server_negotiation.cc:407] Connection header received
0810 14:37:34.867032 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:34.867042 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:34.867155 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:34.867275 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:34.868257 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.869548 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.872282 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.873094 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.873265 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:34.877528 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:34.877539 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:34.877546 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:34.877598 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:34.883154 (+ 5556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:34.884321 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:34.884326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:34.884328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:34.884491 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:34.885107 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:34.885114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:34.885117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:34.885776 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:34.886017 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:34.886795 (+ 778us) server_negotiation.cc:299] Negotiation successful
0810 14:37:34.887102 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260810 14:37:34.887766 24195 heartbeater.cc:347] Connected to a master server at 127.21.55.126:46841
I20260810 14:37:34.888005 24195 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:34.888286 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.866012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46841 (local address 127.21.55.67:54525)
0810 14:37:34.866671 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:34.866698 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:34.866877 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:34.867325 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:34.867330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:34.867793 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:34.868105 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:34.868114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.869764 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.869772 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:34.872019 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:34.872045 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.873372 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.873382 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:34.873655 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:34.875086 (+ 1431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:34.875214 (+ 128us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:34.877196 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:34.883408 (+ 6212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:34.883416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:34.883421 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:34.884151 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:34.884630 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:34.884639 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:34.884644 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:34.884822 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:34.885988 (+ 1166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:34.885997 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:34.886343 (+ 346us) client_negotiation.cc:769] Sending connection context
0810 14:37:34.886592 (+ 249us) client_negotiation.cc:240] Negotiation successful
0810 14:37:34.887151 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":335,"threads_started":1}
I20260810 14:37:34.889499 24195 heartbeater.cc:511] Master 127.21.55.126:46841 requested a full tablet report, sending...
I20260810 14:37:34.891474 23487 ts_manager.cc:194] Registered new tserver with Master: fd42b6727a734237afa13f05656e9591 (127.21.55.67:44377)
I20260810 14:37:34.891788 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23989
I20260810 14:37:34.891985 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:37:34.892544 21725 external_mini_cluster.cc:1428] Setting key 27e24841b05ae575a2a4e05e101771fd
I20260810 14:37:34.892688 23487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:52035
I20260810 14:37:34.896899 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:37:34 dist-test-slave-h6n0 krb5kdc[23419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372654, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:37:34 dist-test-slave-h6n0 krb5kdc[23419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372654, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:34.957746 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.940885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48158 (local address 127.21.55.126:46841)
0810 14:37:34.941085 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:34.941090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:34.941579 (+ 489us) server_negotiation.cc:407] Connection header received
0810 14:37:34.941855 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:34.941862 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:34.941995 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:34.942144 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:34.943916 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.944765 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.946127 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:34.946625 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:34.946768 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:34.951061 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:34.951067 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:34.951071 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:34.951107 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:34.955004 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:34.955694 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:34.955700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:34.955702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:34.955833 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:34.956241 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:34.956246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:34.956249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:34.956545 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:34.956848 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:34.957155 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 14:37:34.957392 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:37:34.963625 23487 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:37:34.972393 23487 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:37:35.010840 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.988594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.21.55.65:37417)
0810 14:37:35.001827 (+ 13233us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:35.001836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:35.001876 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:37:35.001974 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:35.001981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:35.002296 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:35.002479 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:35.003795 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.005360 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.008391 (+ 3031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.009642 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.009812 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.010024 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:35.010318 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 14:37:35.010481 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13003,"thread_start_us":12841,"threads_started":1}
I20260810 14:37:35.010883 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.988547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:37417 (local address 127.0.0.1:50762)
0810 14:37:35.001156 (+ 12609us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:35.001176 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:35.001294 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:35.002636 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:35.002645 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:35.002692 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:35.003390 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.003412 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.005485 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.005492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.008191 (+ 2699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.008207 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.009847 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.009854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.010060 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.010109 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 14:37:35.010263 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 14:37:35.010451 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12485,"mutex_wait_us":183,"spinlock_wait_cycles":896,"thread_start_us":11626,"threads_started":1}
I20260810 14:37:35.012530 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.988000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:46853 (local address 127.0.0.1:42796)
0810 14:37:35.000786 (+ 12786us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:35.000855 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:35.001145 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:35.002636 (+ 1491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:35.002645 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:35.002698 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:35.003974 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.003985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.005987 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.005993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.009002 (+ 3009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.009016 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.011247 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.011255 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.011459 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.011495 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 14:37:35.011578 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 14:37:35.011725 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12365,"mutex_wait_us":698,"spinlock_wait_cycles":3968,"thread_start_us":13392,"threads_started":1}
I20260810 14:37:35.013273 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.001728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57968 (local address 127.21.55.67:44377)
0810 14:37:35.003675 (+ 1947us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:35.003686 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:35.004198 (+ 512us) server_negotiation.cc:407] Connection header received
0810 14:37:35.004312 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:35.004319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:35.004564 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:35.004763 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:35.005371 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.006804 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.009911 (+ 3107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.011680 (+ 1769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.012000 (+ 320us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.012507 (+ 507us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:35.012648 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 14:37:35.012887 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1767,"thread_start_us":1580,"threads_started":1}
I20260810 14:37:35.014060 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:34.988314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42796 (local address 127.21.55.66:46853)
0810 14:37:35.000080 (+ 11766us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:35.000095 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:35.001642 (+ 1547us) server_negotiation.cc:407] Connection header received
0810 14:37:35.001770 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:35.001779 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:35.002101 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:35.002332 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:35.004199 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.005836 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.009185 (+ 3349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.010756 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.010902 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.013059 (+ 2157us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:35.013275 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 14:37:35.013823 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11504,"thread_start_us":11353,"threads_started":1}
I20260810 14:37:35.012929 24224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.001779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:44377 (local address 127.0.0.1:57968)
0810 14:37:35.004012 (+ 2233us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:35.004031 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:35.004162 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:35.004771 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:35.004776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:35.004801 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:35.005147 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.005157 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.006958 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.006964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.009690 (+ 2726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.009707 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.011969 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.011976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.012214 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.012254 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 14:37:35.012348 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 14:37:35.012540 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2073,"thread_start_us":1908,"threads_started":1}
I20260810 14:37:35.015172 23697 tablet_service.cc:1467] Processing CreateTablet for tablet 8af4fc4cf0e94ecf8854a98fa6207529 (DEFAULT_TABLE table=test-table [id=323ef7fa5339474f93ffb2ee4db0508b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:35.016726 23697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af4fc4cf0e94ecf8854a98fa6207529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:35.018115 23894 tablet_service.cc:1467] Processing CreateTablet for tablet 8af4fc4cf0e94ecf8854a98fa6207529 (DEFAULT_TABLE table=test-table [id=323ef7fa5339474f93ffb2ee4db0508b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:35.019582 23894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af4fc4cf0e94ecf8854a98fa6207529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:35.022387 24089 tablet_service.cc:1467] Processing CreateTablet for tablet 8af4fc4cf0e94ecf8854a98fa6207529 (DEFAULT_TABLE table=test-table [id=323ef7fa5339474f93ffb2ee4db0508b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:35.024024 24089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af4fc4cf0e94ecf8854a98fa6207529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:35.032354 24227 tablet_bootstrap.cc:492] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: Bootstrap starting.
I20260810 14:37:35.034312 24228 tablet_bootstrap.cc:492] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: Bootstrap starting.
I20260810 14:37:35.036525 24229 tablet_bootstrap.cc:492] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: Bootstrap starting.
I20260810 14:37:35.036989 24227 tablet_bootstrap.cc:654] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:35.038262 24227 log.cc:824] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:35.038285 24228 tablet_bootstrap.cc:654] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:35.039615 24228 log.cc:824] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:35.040660 24229 tablet_bootstrap.cc:654] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:35.041429 24227 tablet_bootstrap.cc:492] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: No bootstrap required, opened a new log
I20260810 14:37:35.041718 24227 ts_tablet_manager.cc:1397] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260810 14:37:35.041916 24229 log.cc:824] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:35.043013 24228 tablet_bootstrap.cc:492] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: No bootstrap required, opened a new log
I20260810 14:37:35.043303 24228 ts_tablet_manager.cc:1397] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 14:37:35.044423 24229 tablet_bootstrap.cc:492] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: No bootstrap required, opened a new log
I20260810 14:37:35.044694 24229 ts_tablet_manager.cc:1397] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 14:37:35.053491 24227 raft_consensus.cc:348] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.054015 24227 raft_consensus.cc:374] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:35.054116 24227 raft_consensus.cc:729] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f95c371d9a4f9990154f0c9535202b, State: Initialized, Role: FOLLOWER
I20260810 14:37:35.053776 24229 raft_consensus.cc:348] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.054399 24229 raft_consensus.cc:374] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:35.054507 24229 raft_consensus.cc:729] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd42b6727a734237afa13f05656e9591, State: Initialized, Role: FOLLOWER
I20260810 14:37:35.054968 24227 consensus_queue.cc:260] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.054821 24228 raft_consensus.cc:348] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.055289 24228 raft_consensus.cc:374] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:35.055401 24228 raft_consensus.cc:729] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a901aa5bf1248c5972e6c1a7638700d, State: Initialized, Role: FOLLOWER
I20260810 14:37:35.055330 24229 consensus_queue.cc:260] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.056080 24228 consensus_queue.cc:260] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.057660 24227 ts_tablet_manager.cc:1428] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: Time spent starting tablet: real 0.016s user 0.008s sys 0.010s
I20260810 14:37:35.059005 24195 heartbeater.cc:503] Master 127.21.55.126:46841 was elected leader, sending a full tablet report...
I20260810 14:37:35.059542 23984 heartbeater.cc:503] Master 127.21.55.126:46841 was elected leader, sending a full tablet report...
I20260810 14:37:35.060215 24228 ts_tablet_manager.cc:1428] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260810 14:37:35.061386 24229 ts_tablet_manager.cc:1428] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: Time spent starting tablet: real 0.017s user 0.014s sys 0.001s
W20260810 14:37:35.139824 24197 tablet.cc:2369] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:37:35.186509 23790 tablet.cc:2369] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:37:35.287914 23985 tablet.cc:2369] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:37:35.306460 24236 raft_consensus.cc:482] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:37:35.306840 24236 raft_consensus.cc:504] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.308878 24236 leader_election.cc:288] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6f95c371d9a4f9990154f0c9535202b (127.21.55.65:37417), fd42b6727a734237afa13f05656e9591 (127.21.55.67:44377)
I20260810 14:37:35.316038 24221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.309344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:47799 (local address 127.21.55.65:37417)
0810 14:37:35.309568 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:35.309572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:35.310211 (+ 639us) server_negotiation.cc:407] Connection header received
0810 14:37:35.310623 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:35.310630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:35.310729 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:35.310792 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:35.311503 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.312352 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.314261 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.315089 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.315270 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.315349 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:35.315652 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 14:37:35.315775 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 14:37:35.316231 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.309240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:37417 (local address 127.21.55.66:47799)
0810 14:37:35.310092 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:35.310114 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:35.310361 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:35.310986 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:35.310990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:35.311037 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:35.311358 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.311366 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.312485 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.312492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.314100 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.314112 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.315312 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.315318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.315489 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.315516 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 14:37:35.315628 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 14:37:35.315758 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":46,"thread_start_us":410,"threads_started":1}
I20260810 14:37:35.316237 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.310406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:60279 (local address 127.21.55.67:44377)
0810 14:37:35.310644 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:35.310652 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:35.310889 (+ 237us) server_negotiation.cc:407] Connection header received
0810 14:37:35.311084 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:35.311089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:35.311204 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:35.311310 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:35.311899 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.312667 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.314472 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.315290 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.315470 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.315563 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:35.315839 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 14:37:35.315963 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:37:35.316820 24234 raft_consensus.cc:482] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:37:35.317063 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.310165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:44377 (local address 127.21.55.66:60279)
0810 14:37:35.310787 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:35.310801 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:35.310949 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:35.311330 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:35.311334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:35.311363 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:35.311751 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.311761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.312813 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.312820 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.314308 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.314320 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.315522 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.315528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.315680 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.315702 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 14:37:35.315811 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 14:37:35.315937 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":1,"thread_start_us":338,"threads_started":1}
I20260810 14:37:35.317131 24234 raft_consensus.cc:504] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.317766 23725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af4fc4cf0e94ecf8854a98fa6207529" candidate_uuid: "9a901aa5bf1248c5972e6c1a7638700d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6f95c371d9a4f9990154f0c9535202b" is_pre_election: true
I20260810 14:37:35.318276 23725 raft_consensus.cc:2454] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a901aa5bf1248c5972e6c1a7638700d in term 0.
I20260810 14:37:35.318579 24118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af4fc4cf0e94ecf8854a98fa6207529" candidate_uuid: "9a901aa5bf1248c5972e6c1a7638700d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd42b6727a734237afa13f05656e9591" is_pre_election: true
I20260810 14:37:35.319124 23823 leader_election.cc:302] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a901aa5bf1248c5972e6c1a7638700d, c6f95c371d9a4f9990154f0c9535202b; no voters:
I20260810 14:37:35.319247 24234 leader_election.cc:288] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a901aa5bf1248c5972e6c1a7638700d (127.21.55.66:46853), fd42b6727a734237afa13f05656e9591 (127.21.55.67:44377)
I20260810 14:37:35.319269 24118 raft_consensus.cc:2454] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a901aa5bf1248c5972e6c1a7638700d in term 0.
I20260810 14:37:35.319553 24236 raft_consensus.cc:2781] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:37:35.319700 24236 raft_consensus.cc:482] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:37:35.319808 24236 raft_consensus.cc:3037] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:35.323763 24236 raft_consensus.cc:504] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.324556 24236 leader_election.cc:288] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [CANDIDATE]: Term 1 election: Requested vote from peers c6f95c371d9a4f9990154f0c9535202b (127.21.55.65:37417), fd42b6727a734237afa13f05656e9591 (127.21.55.67:44377)
I20260810 14:37:35.325038 23725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af4fc4cf0e94ecf8854a98fa6207529" candidate_uuid: "9a901aa5bf1248c5972e6c1a7638700d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6f95c371d9a4f9990154f0c9535202b"
I20260810 14:37:35.325250 23725 raft_consensus.cc:3037] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:35.328397 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.320063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:57645 (local address 127.21.55.66:46853)
0810 14:37:35.320331 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:35.320338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:35.320512 (+ 174us) server_negotiation.cc:407] Connection header received
0810 14:37:35.320839 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:35.320845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:35.320941 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:35.321017 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:35.321918 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.323106 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.325866 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.327179 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.327347 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.327422 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:35.327963 (+ 541us) server_negotiation.cc:299] Negotiation successful
0810 14:37:35.328130 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:37:35.328599 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.319526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:46853 (local address 127.21.55.65:57645)
0810 14:37:35.320407 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:35.320429 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:35.320642 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:35.321250 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:35.321255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:35.321284 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:35.321767 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.321778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.323372 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.323377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.325670 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.325692 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.327435 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.327442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.327669 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.327803 (+ 134us) client_negotiation.cc:769] Sending connection context
0810 14:37:35.327921 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 14:37:35.328089 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":22,"thread_start_us":1311,"threads_started":1}
I20260810 14:37:35.328547 23725 raft_consensus.cc:2454] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a901aa5bf1248c5972e6c1a7638700d in term 1.
I20260810 14:37:35.329574 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af4fc4cf0e94ecf8854a98fa6207529" candidate_uuid: "c6f95c371d9a4f9990154f0c9535202b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a901aa5bf1248c5972e6c1a7638700d" is_pre_election: true
I20260810 14:37:35.330199 23921 raft_consensus.cc:2379] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c6f95c371d9a4f9990154f0c9535202b in current term 1: Already voted for candidate 9a901aa5bf1248c5972e6c1a7638700d in this term.
I20260810 14:37:35.330503 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.320002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:44377 (local address 127.21.55.65:60155)
0810 14:37:35.323234 (+ 3232us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:35.323259 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:35.323406 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:35.323818 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:35.323823 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:35.323852 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:35.324587 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.324613 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.326025 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.326038 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.328229 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.328245 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.329780 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.329784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.329967 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.329990 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 14:37:35.330074 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 14:37:35.330167 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3084,"mutex_wait_us":24,"thread_start_us":2847,"threads_started":1}
I20260810 14:37:35.330621 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.320582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:60155 (local address 127.21.55.67:44377)
0810 14:37:35.320839 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:35.320844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:35.323375 (+ 2531us) server_negotiation.cc:407] Connection header received
0810 14:37:35.323564 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:35.323571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:35.323657 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:35.323777 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:35.324794 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.325869 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.328425 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.329481 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.329578 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.329622 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:35.330141 (+ 519us) server_negotiation.cc:299] Negotiation successful
0810 14:37:35.330297 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260810 14:37:35.331185 23823 leader_election.cc:302] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a901aa5bf1248c5972e6c1a7638700d, c6f95c371d9a4f9990154f0c9535202b; no voters:
I20260810 14:37:35.331571 24236 raft_consensus.cc:2781] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:35.331513 24118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af4fc4cf0e94ecf8854a98fa6207529" candidate_uuid: "c6f95c371d9a4f9990154f0c9535202b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd42b6727a734237afa13f05656e9591" is_pre_election: true
I20260810 14:37:35.331537 24116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af4fc4cf0e94ecf8854a98fa6207529" candidate_uuid: "9a901aa5bf1248c5972e6c1a7638700d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd42b6727a734237afa13f05656e9591"
I20260810 14:37:35.331836 24118 raft_consensus.cc:2454] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6f95c371d9a4f9990154f0c9535202b in term 0.
I20260810 14:37:35.332190 24236 raft_consensus.cc:686] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 1 LEADER]: Becoming Leader. State: Replica: 9a901aa5bf1248c5972e6c1a7638700d, State: Running, Role: LEADER
I20260810 14:37:35.332327 23621 leader_election.cc:302] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c6f95c371d9a4f9990154f0c9535202b, fd42b6727a734237afa13f05656e9591; no voters: 9a901aa5bf1248c5972e6c1a7638700d
I20260810 14:37:35.332805 24234 raft_consensus.cc:2741] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 14:37:35.332743 24236 consensus_queue.cc:237] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.337868 23485 catalog_manager.cc:5495] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a901aa5bf1248c5972e6c1a7638700d (127.21.55.66). New cstate: current_term: 1 leader_uuid: "9a901aa5bf1248c5972e6c1a7638700d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } health_report { overall_health: UNKNOWN } } }
I20260810 14:37:35.384729 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.378132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.21.55.66:46853)
0810 14:37:35.378312 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:35.378316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:35.378336 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:37:35.378884 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:35.378890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:35.379007 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:35.379187 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:35.379623 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.380660 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.382206 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.382722 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.382822 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.383357 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:35.383976 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:35.384096 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:35.384392 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 14:37:35.384481 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:37:35.394713 23725 raft_consensus.cc:1261] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 1 FOLLOWER]: Refusing update from remote peer 9a901aa5bf1248c5972e6c1a7638700d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:35.395227 24116 raft_consensus.cc:3037] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:35.395891 24241 consensus_queue.cc:1035] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:35.398262 24116 raft_consensus.cc:1261] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591 [term 1 FOLLOWER]: Refusing update from remote peer 9a901aa5bf1248c5972e6c1a7638700d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:35.399137 24241 consensus_queue.cc:1035] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:35.438848 23485 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:35.439168 23485 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:35.443387 23485 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900: Sending DeleteTablet for 3 replicas of tablet 8af4fc4cf0e94ecf8854a98fa6207529
I20260810 14:37:35.444794 23894 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af4fc4cf0e94ecf8854a98fa6207529 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:35 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:42796
I20260810 14:37:35.446465 24089 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af4fc4cf0e94ecf8854a98fa6207529 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:35 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:57968
I20260810 14:37:35.446484 23485 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 14:37:35.447007 23697 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af4fc4cf0e94ecf8854a98fa6207529 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:35 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:50762
W20260810 14:37:35.447296 23485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:37:35.449352 24264 tablet_replica.cc:330] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: stopping tablet replica
I20260810 14:37:35.449961 24264 raft_consensus.cc:2229] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:35.450569 24264 raft_consensus.cc:2258] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:35.453883 24264 ts_tablet_manager.cc:1905] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:35.456332 24261 tablet_replica.cc:330] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: stopping tablet replica
I20260810 14:37:35.458962 23696 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc3ead0b04e43b79d92c74586a2bb6e (DEFAULT_TABLE table=test-table [id=99944a1f71924807822b9c1d0cbfdd9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:35.459766 23696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc3ead0b04e43b79d92c74586a2bb6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:35.459968 23894 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc3ead0b04e43b79d92c74586a2bb6e (DEFAULT_TABLE table=test-table [id=99944a1f71924807822b9c1d0cbfdd9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:35.460494 23894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc3ead0b04e43b79d92c74586a2bb6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:35.461457 24265 tablet_replica.cc:330] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: stopping tablet replica
I20260810 14:37:35.461514 24089 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc3ead0b04e43b79d92c74586a2bb6e (DEFAULT_TABLE table=test-table [id=99944a1f71924807822b9c1d0cbfdd9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:35.461961 24089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc3ead0b04e43b79d92c74586a2bb6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:35.461961 24265 raft_consensus.cc:2229] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:35.462555 24265 raft_consensus.cc:2258] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:35.467303 24227 tablet_bootstrap.cc:492] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b: Bootstrap starting.
I20260810 14:37:35.467434 24261 raft_consensus.cc:2229] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:37:35.468544 24229 tablet_bootstrap.cc:492] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591: Bootstrap starting.
I20260810 14:37:35.469267 24227 tablet_bootstrap.cc:654] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:35.470696 24229 tablet_bootstrap.cc:654] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:35.470978 24228 tablet_bootstrap.cc:492] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d: Bootstrap starting.
I20260810 14:37:35.471073 24264 ts_tablet_manager.cc:1918] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:37:35.471304 24264 log.cc:1196] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/8af4fc4cf0e94ecf8854a98fa6207529
I20260810 14:37:35.471961 24264 ts_tablet_manager.cc:1939] T 8af4fc4cf0e94ecf8854a98fa6207529 P fd42b6727a734237afa13f05656e9591: Deleting consensus metadata
I20260810 14:37:35.471573 24261 raft_consensus.cc:2258] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:35.473206 24228 tablet_bootstrap.cc:654] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:35.473893 23468 catalog_manager.cc:4841] TS fd42b6727a734237afa13f05656e9591 (127.21.55.67:44377): tablet 8af4fc4cf0e94ecf8854a98fa6207529 (table test-table [id=323ef7fa5339474f93ffb2ee4db0508b]) successfully deleted
I20260810 14:37:35.476706 24261 ts_tablet_manager.cc:1905] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:35.482156 24229 tablet_bootstrap.cc:492] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591: No bootstrap required, opened a new log
I20260810 14:37:35.482352 24229 ts_tablet_manager.cc:1397] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260810 14:37:35.483356 24229 raft_consensus.cc:348] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.483631 24229 raft_consensus.cc:374] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:35.483758 24229 raft_consensus.cc:729] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd42b6727a734237afa13f05656e9591, State: Initialized, Role: FOLLOWER
I20260810 14:37:35.484112 24229 consensus_queue.cc:260] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.484601 24261 ts_tablet_manager.cc:1918] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:37:35.484778 24261 log.cc:1196] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/8af4fc4cf0e94ecf8854a98fa6207529
I20260810 14:37:35.485327 24261 ts_tablet_manager.cc:1939] T 8af4fc4cf0e94ecf8854a98fa6207529 P 9a901aa5bf1248c5972e6c1a7638700d: Deleting consensus metadata
I20260810 14:37:35.487129 24229 ts_tablet_manager.cc:1428] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260810 14:37:35.487329 24265 ts_tablet_manager.cc:1905] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:35.488437 24227 tablet_bootstrap.cc:492] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b: No bootstrap required, opened a new log
I20260810 14:37:35.488649 24227 ts_tablet_manager.cc:1397] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.004s
I20260810 14:37:35.489825 24227 raft_consensus.cc:348] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.490128 24228 tablet_bootstrap.cc:492] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d: No bootstrap required, opened a new log
I20260810 14:37:35.490165 24227 raft_consensus.cc:374] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:35.490306 24228 ts_tablet_manager.cc:1397] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.005s
I20260810 14:37:35.490306 24227 raft_consensus.cc:729] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f95c371d9a4f9990154f0c9535202b, State: Initialized, Role: FOLLOWER
I20260810 14:37:35.490628 24227 consensus_queue.cc:260] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.491205 24227 ts_tablet_manager.cc:1428] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:37:35.491297 24228 raft_consensus.cc:348] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.491552 24228 raft_consensus.cc:374] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:35.491667 24228 raft_consensus.cc:729] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a901aa5bf1248c5972e6c1a7638700d, State: Initialized, Role: FOLLOWER
I20260810 14:37:35.492003 24228 consensus_queue.cc:260] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.492336 23466 catalog_manager.cc:4841] TS 9a901aa5bf1248c5972e6c1a7638700d (127.21.55.66:46853): tablet 8af4fc4cf0e94ecf8854a98fa6207529 (table test-table [id=323ef7fa5339474f93ffb2ee4db0508b]) successfully deleted
I20260810 14:37:35.495234 24228 ts_tablet_manager.cc:1428] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260810 14:37:35.498070 24265 ts_tablet_manager.cc:1918] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:37:35.498239 24265 log.cc:1196] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/8af4fc4cf0e94ecf8854a98fa6207529
I20260810 14:37:35.498786 24265 ts_tablet_manager.cc:1939] T 8af4fc4cf0e94ecf8854a98fa6207529 P c6f95c371d9a4f9990154f0c9535202b: Deleting consensus metadata
I20260810 14:37:35.500334 23468 catalog_manager.cc:4841] TS c6f95c371d9a4f9990154f0c9535202b (127.21.55.65:37417): tablet 8af4fc4cf0e94ecf8854a98fa6207529 (table test-table [id=323ef7fa5339474f93ffb2ee4db0508b]) successfully deleted
I20260810 14:37:35.668768 24234 raft_consensus.cc:482] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:37:35.669018 24234 raft_consensus.cc:504] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.669981 24234 leader_election.cc:288] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a901aa5bf1248c5972e6c1a7638700d (127.21.55.66:46853), fd42b6727a734237afa13f05656e9591 (127.21.55.67:44377)
I20260810 14:37:35.670697 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc3ead0b04e43b79d92c74586a2bb6e" candidate_uuid: "c6f95c371d9a4f9990154f0c9535202b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a901aa5bf1248c5972e6c1a7638700d" is_pre_election: true
I20260810 14:37:35.670686 24116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc3ead0b04e43b79d92c74586a2bb6e" candidate_uuid: "c6f95c371d9a4f9990154f0c9535202b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd42b6727a734237afa13f05656e9591" is_pre_election: true
I20260810 14:37:35.670990 24116 raft_consensus.cc:2454] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6f95c371d9a4f9990154f0c9535202b in term 0.
I20260810 14:37:35.671204 23921 raft_consensus.cc:2454] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6f95c371d9a4f9990154f0c9535202b in term 0.
I20260810 14:37:35.671439 23621 leader_election.cc:302] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6f95c371d9a4f9990154f0c9535202b, fd42b6727a734237afa13f05656e9591; no voters:
I20260810 14:37:35.671766 24234 raft_consensus.cc:2781] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:37:35.671903 24234 raft_consensus.cc:482] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:37:35.672039 24234 raft_consensus.cc:3037] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:35.675527 24234 raft_consensus.cc:504] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.676383 24234 leader_election.cc:288] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [CANDIDATE]: Term 1 election: Requested vote from peers 9a901aa5bf1248c5972e6c1a7638700d (127.21.55.66:46853), fd42b6727a734237afa13f05656e9591 (127.21.55.67:44377)
I20260810 14:37:35.677179 23921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc3ead0b04e43b79d92c74586a2bb6e" candidate_uuid: "c6f95c371d9a4f9990154f0c9535202b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a901aa5bf1248c5972e6c1a7638700d"
I20260810 14:37:35.677248 24116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc3ead0b04e43b79d92c74586a2bb6e" candidate_uuid: "c6f95c371d9a4f9990154f0c9535202b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd42b6727a734237afa13f05656e9591"
I20260810 14:37:35.677462 23921 raft_consensus.cc:3037] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:35.677572 24116 raft_consensus.cc:3037] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:35.681538 24116 raft_consensus.cc:2454] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6f95c371d9a4f9990154f0c9535202b in term 1.
I20260810 14:37:35.681766 23921 raft_consensus.cc:2454] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6f95c371d9a4f9990154f0c9535202b in term 1.
I20260810 14:37:35.682471 23621 leader_election.cc:302] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6f95c371d9a4f9990154f0c9535202b, fd42b6727a734237afa13f05656e9591; no voters:
I20260810 14:37:35.682994 24234 raft_consensus.cc:2781] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:35.683574 24234 raft_consensus.cc:686] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 1 LEADER]: Becoming Leader. State: Replica: c6f95c371d9a4f9990154f0c9535202b, State: Running, Role: LEADER
I20260810 14:37:35.684127 24234 consensus_queue.cc:237] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.689071 23485 catalog_manager.cc:5495] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b reported cstate change: term changed from 0 to 1, leader changed from <none> to c6f95c371d9a4f9990154f0c9535202b (127.21.55.65). New cstate: current_term: 1 leader_uuid: "c6f95c371d9a4f9990154f0c9535202b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } health_report { overall_health: UNKNOWN } } }
Aug 10 14:37:35 dist-test-slave-h6n0 krb5kdc[23419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372652, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:35.742980 24285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.725548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48164 (local address 127.21.55.126:46841)
0810 14:37:35.726577 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:35.726583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:35.726610 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:37:35.726651 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:35.726654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:35.726712 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:35.726826 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:35.727911 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.728738 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.729984 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.730397 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.730557 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.735150 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:35.735155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:35.735159 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:35.735193 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:35.739020 (+ 3827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:35.740184 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:35.740190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:35.740198 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:35.740364 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:35.740837 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:35.740845 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:35.740849 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:35.741358 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:35.741574 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:35.742117 (+ 543us) server_negotiation.cc:299] Negotiation successful
0810 14:37:35.742390 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":701,"threads_started":1}
I20260810 14:37:35.743072 24283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:35.725470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46841 (local address 127.0.0.1:48164)
0810 14:37:35.726223 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:35.726255 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:35.726445 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:35.726849 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:35.726852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:35.727558 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:35.727758 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.727767 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.728875 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.728882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.729828 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:35.729839 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:35.730588 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:35.730594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:35.730781 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:35.731657 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:35.731676 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:35.734834 (+ 3158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:35.739247 (+ 4413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:35.739272 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:35.739283 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:35.740008 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:35.740498 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:35.740504 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:35.740508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:35.740708 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:35.741540 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:35.741552 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:35.741794 (+ 242us) client_negotiation.cc:769] Sending connection context
0810 14:37:35.742010 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 14:37:35.742602 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":179,"thread_start_us":400,"threads_started":1}
I20260810 14:37:35.751327 23485 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:48164:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 14:37:35.752691 23485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:37:35.761112 23696 tablet_service.cc:1467] Processing CreateTablet for tablet 63df528f0a83457da1c3057ff3335bc2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5d9db178ef3f4524bc491674960ce387]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:37:35.761669 23696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63df528f0a83457da1c3057ff3335bc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:35.762090 24089 tablet_service.cc:1467] Processing CreateTablet for tablet 63df528f0a83457da1c3057ff3335bc2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5d9db178ef3f4524bc491674960ce387]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:37:35.762921 24089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63df528f0a83457da1c3057ff3335bc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:35.767388 23894 tablet_service.cc:1467] Processing CreateTablet for tablet 63df528f0a83457da1c3057ff3335bc2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5d9db178ef3f4524bc491674960ce387]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:37:35.767969 23894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63df528f0a83457da1c3057ff3335bc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:35.774703 24229 tablet_bootstrap.cc:492] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591: Bootstrap starting.
I20260810 14:37:35.775067 24228 tablet_bootstrap.cc:492] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: Bootstrap starting.
I20260810 14:37:35.776647 24229 tablet_bootstrap.cc:654] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:35.776863 24228 tablet_bootstrap.cc:654] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:35.777726 24227 tablet_bootstrap.cc:492] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b: Bootstrap starting.
I20260810 14:37:35.779882 24227 tablet_bootstrap.cc:654] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:35.788707 24229 tablet_bootstrap.cc:492] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591: No bootstrap required, opened a new log
I20260810 14:37:35.788992 24229 ts_tablet_manager.cc:1397] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.009s
I20260810 14:37:35.790222 24229 raft_consensus.cc:348] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.790637 24229 raft_consensus.cc:374] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:35.790732 24229 raft_consensus.cc:729] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd42b6727a734237afa13f05656e9591, State: Initialized, Role: FOLLOWER
I20260810 14:37:35.791150 24229 consensus_queue.cc:260] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.792205 24235 tablet_replica.cc:439] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } } }
I20260810 14:37:35.792673 24235 tablet_replica.cc:442] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:35.793912 24229 ts_tablet_manager.cc:1428] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260810 14:37:35.795182 24227 tablet_bootstrap.cc:492] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b: No bootstrap required, opened a new log
I20260810 14:37:35.795312 24228 tablet_bootstrap.cc:492] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: No bootstrap required, opened a new log
I20260810 14:37:35.795516 24228 ts_tablet_manager.cc:1397] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.005s
I20260810 14:37:35.795501 24227 ts_tablet_manager.cc:1397] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.015s
I20260810 14:37:35.796394 24228 raft_consensus.cc:348] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.796959 24228 raft_consensus.cc:374] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:35.797118 24228 raft_consensus.cc:729] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a901aa5bf1248c5972e6c1a7638700d, State: Initialized, Role: FOLLOWER
I20260810 14:37:35.797639 24228 consensus_queue.cc:260] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.796387 24227 raft_consensus.cc:348] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.798478 24227 raft_consensus.cc:374] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:35.798499 24236 tablet_replica.cc:439] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } } }
I20260810 14:37:35.798906 24228 ts_tablet_manager.cc:1428] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 14:37:35.798781 24227 raft_consensus.cc:729] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6f95c371d9a4f9990154f0c9535202b, State: Initialized, Role: FOLLOWER
I20260810 14:37:35.798942 24236 tablet_replica.cc:442] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:35.799391 24227 consensus_queue.cc:260] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:35.800061 24234 tablet_replica.cc:439] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } } }
I20260810 14:37:35.800271 24227 ts_tablet_manager.cc:1428] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 14:37:35.800369 24234 tablet_replica.cc:442] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:36.027730 24236 raft_consensus.cc:482] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:37:36.027974 24236 raft_consensus.cc:504] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:36.028627 24236 leader_election.cc:288] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6f95c371d9a4f9990154f0c9535202b (127.21.55.65:37417), fd42b6727a734237afa13f05656e9591 (127.21.55.67:44377)
I20260810 14:37:36.029894 23725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63df528f0a83457da1c3057ff3335bc2" candidate_uuid: "9a901aa5bf1248c5972e6c1a7638700d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6f95c371d9a4f9990154f0c9535202b" is_pre_election: true
I20260810 14:37:36.029891 24116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63df528f0a83457da1c3057ff3335bc2" candidate_uuid: "9a901aa5bf1248c5972e6c1a7638700d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd42b6727a734237afa13f05656e9591" is_pre_election: true
I20260810 14:37:36.030193 23725 raft_consensus.cc:2454] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a901aa5bf1248c5972e6c1a7638700d in term 0.
I20260810 14:37:36.030193 24116 raft_consensus.cc:2454] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a901aa5bf1248c5972e6c1a7638700d in term 0.
I20260810 14:37:36.030871 23823 leader_election.cc:302] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a901aa5bf1248c5972e6c1a7638700d, fd42b6727a734237afa13f05656e9591; no voters:
I20260810 14:37:36.031351 24236 raft_consensus.cc:2781] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:37:36.031507 24236 raft_consensus.cc:482] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:37:36.031646 24236 raft_consensus.cc:3037] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:36.034368 24236 raft_consensus.cc:504] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:36.035435 24236 leader_election.cc:288] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [CANDIDATE]: Term 1 election: Requested vote from peers c6f95c371d9a4f9990154f0c9535202b (127.21.55.65:37417), fd42b6727a734237afa13f05656e9591 (127.21.55.67:44377)
I20260810 14:37:36.035951 23725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63df528f0a83457da1c3057ff3335bc2" candidate_uuid: "9a901aa5bf1248c5972e6c1a7638700d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6f95c371d9a4f9990154f0c9535202b"
I20260810 14:37:36.036230 23725 raft_consensus.cc:3037] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:36.036226 24116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63df528f0a83457da1c3057ff3335bc2" candidate_uuid: "9a901aa5bf1248c5972e6c1a7638700d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd42b6727a734237afa13f05656e9591"
I20260810 14:37:36.036449 24116 raft_consensus.cc:3037] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:36.039448 23725 raft_consensus.cc:2454] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a901aa5bf1248c5972e6c1a7638700d in term 1.
I20260810 14:37:36.039448 24116 raft_consensus.cc:2454] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a901aa5bf1248c5972e6c1a7638700d in term 1.
I20260810 14:37:36.039978 23823 leader_election.cc:302] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a901aa5bf1248c5972e6c1a7638700d, fd42b6727a734237afa13f05656e9591; no voters:
I20260810 14:37:36.040376 24236 raft_consensus.cc:2781] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:36.040920 24236 raft_consensus.cc:686] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [term 1 LEADER]: Becoming Leader. State: Replica: 9a901aa5bf1248c5972e6c1a7638700d, State: Running, Role: LEADER
I20260810 14:37:36.041313 24236 consensus_queue.cc:237] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } }
I20260810 14:37:36.043300 24293 tablet_replica.cc:439] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: TxnStatusTablet state changed. Reason: New leader 9a901aa5bf1248c5972e6c1a7638700d. Latest consensus state: current_term: 1 leader_uuid: "9a901aa5bf1248c5972e6c1a7638700d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } } }
I20260810 14:37:36.043776 24293 tablet_replica.cc:442] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:37:36.045308 24297 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 14:37:36.044982 23486 catalog_manager.cc:5495] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a901aa5bf1248c5972e6c1a7638700d (127.21.55.66). New cstate: current_term: 1 leader_uuid: "9a901aa5bf1248c5972e6c1a7638700d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } health_report { overall_health: UNKNOWN } } }
I20260810 14:37:36.045660 24297 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 14:37:36.092185 24283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.086389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:46853 (local address 127.0.0.1:42810)
0810 14:37:36.086659 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:36.086682 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:36.086879 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:36.087533 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:36.087538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:36.087554 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:37:36.087782 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:36.087792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.088961 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.088966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:36.090020 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:36.090031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.090894 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.090903 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:36.091053 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.091107 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:37:36.091549 (+ 442us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:37:36.091575 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 14:37:36.091674 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 14:37:36.091841 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 14:37:36.092231 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.086487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42810 (local address 127.21.55.66:46853)
0810 14:37:36.087218 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:36.087222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:36.087237 (+ 15us) server_negotiation.cc:407] Connection header received
0810 14:37:36.087304 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:36.087308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:36.087383 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:36.087504 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:36.087973 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.088806 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.090169 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.090708 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.090854 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.091329 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:36.091434 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:36.091547 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:36.091712 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 14:37:36.091794 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":368,"threads_started":1}
I20260810 14:37:36.098665 23725 raft_consensus.cc:1261] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b [term 1 FOLLOWER]: Refusing update from remote peer 9a901aa5bf1248c5972e6c1a7638700d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:36.098763 24116 raft_consensus.cc:1261] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591 [term 1 FOLLOWER]: Refusing update from remote peer 9a901aa5bf1248c5972e6c1a7638700d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:36.099704 24293 consensus_queue.cc:1035] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:36.100281 24236 consensus_queue.cc:1035] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:36.108206 24234 tablet_replica.cc:439] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b: TxnStatusTablet state changed. Reason: New leader 9a901aa5bf1248c5972e6c1a7638700d. Latest consensus state: current_term: 1 leader_uuid: "9a901aa5bf1248c5972e6c1a7638700d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } } }
I20260810 14:37:36.108541 24234 tablet_replica.cc:442] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:36.119701 24236 tablet_replica.cc:439] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a901aa5bf1248c5972e6c1a7638700d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } } }
I20260810 14:37:36.120095 24236 tablet_replica.cc:442] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:37:36.121047 24234 tablet_replica.cc:439] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a901aa5bf1248c5972e6c1a7638700d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } } }
I20260810 14:37:36.121352 24234 tablet_replica.cc:442] T 63df528f0a83457da1c3057ff3335bc2 P c6f95c371d9a4f9990154f0c9535202b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:36.124292 24235 tablet_replica.cc:439] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591: TxnStatusTablet state changed. Reason: New leader 9a901aa5bf1248c5972e6c1a7638700d. Latest consensus state: current_term: 1 leader_uuid: "9a901aa5bf1248c5972e6c1a7638700d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } } }
I20260810 14:37:36.124642 24235 tablet_replica.cc:442] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:36.128358 24303 mvcc.cc:205] Tried to move back new op lower bound from 7316982399371341824 to 7316982399148249088. Current Snapshot: MvccSnapshot[applied={T|T < 7316982399371341824}]
I20260810 14:37:36.128284 24293 tablet_replica.cc:439] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a901aa5bf1248c5972e6c1a7638700d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } } }
I20260810 14:37:36.128825 24293 tablet_replica.cc:442] T 63df528f0a83457da1c3057ff3335bc2 P 9a901aa5bf1248c5972e6c1a7638700d: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:37:36.137262 24235 tablet_replica.cc:439] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a901aa5bf1248c5972e6c1a7638700d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6f95c371d9a4f9990154f0c9535202b" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37417 } } peers { permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 } } peers { permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 } } }
I20260810 14:37:36.137550 24235 tablet_replica.cc:442] T 63df528f0a83457da1c3057ff3335bc2 P fd42b6727a734237afa13f05656e9591: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:36.147117 24234 consensus_queue.cc:1035] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a901aa5bf1248c5972e6c1a7638700d" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 46853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:36.156000 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.143956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50774 (local address 127.21.55.65:37417)
0810 14:37:36.144909 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:36.144917 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:36.145773 (+ 856us) server_negotiation.cc:407] Connection header received
0810 14:37:36.146302 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:36.146311 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:36.146466 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:36.146709 (+ 243us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:36.147456 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.148814 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.151939 (+ 3125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.152590 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.152744 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.153388 (+ 644us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:36.154190 (+ 802us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:36.154355 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:36.155056 (+ 701us) server_negotiation.cc:299] Negotiation successful
0810 14:37:36.155151 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":640,"threads_started":1}
I20260810 14:37:36.157454 24234 consensus_queue.cc:1035] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd42b6727a734237afa13f05656e9591" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:36.181466 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.174101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:46853 (local address 127.21.55.65:45337)
0810 14:37:36.174989 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:36.175026 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:36.175216 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:36.175694 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:36.175699 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:36.175714 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:37:36.176053 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:36.176064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.177750 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.177754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:36.179096 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:36.179109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.180251 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.180256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:36.180390 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.180455 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:37:36.180928 (+ 473us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:37:36.180938 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:37:36.181025 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 14:37:36.181145 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":645,"threads_started":1}
I20260810 14:37:36.181473 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.174683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:45337 (local address 127.21.55.66:46853)
0810 14:37:36.174941 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:36.174949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:36.175148 (+ 199us) server_negotiation.cc:407] Connection header received
0810 14:37:36.175387 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:36.175394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:36.175492 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:36.175619 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:36.176222 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.177553 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.179250 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.180055 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.180207 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.180659 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:36.180803 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:36.180926 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:36.181076 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 14:37:36.181168 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 14:37:36.236744 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.228712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:49767 (local address 127.21.55.65:37417)
0810 14:37:36.228894 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:36.228900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:36.229736 (+ 836us) server_negotiation.cc:407] Connection header received
0810 14:37:36.229910 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:36.229916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:36.230010 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:36.230114 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:36.230796 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.231828 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.233805 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.234472 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.234627 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.235808 (+ 1181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:36.235941 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:36.236052 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:36.236279 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 14:37:36.236364 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:37:36.237358 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.228744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:37417 (local address 127.21.55.66:49767)
0810 14:37:36.229609 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:36.229631 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:36.229775 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:36.230165 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:36.230168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:36.230179 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:37:36.230508 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:36.230517 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.232024 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.232029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:36.233660 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:36.233674 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.235003 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.235014 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:36.235378 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.235534 (+ 156us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:37:36.236095 (+ 561us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:37:36.236117 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 14:37:36.236236 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 14:37:36.236440 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":416,"threads_started":1}
I20260810 14:37:36.291460 23485 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:36.291747 23485 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:36.295531 23485 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af26d05611f74455b859c52c01d2a900: Sending DeleteTablet for 3 replicas of tablet 4cc3ead0b04e43b79d92c74586a2bb6e
I20260810 14:37:36.297818 24089 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc3ead0b04e43b79d92c74586a2bb6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:36 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:57968
I20260810 14:37:36.298895 23696 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc3ead0b04e43b79d92c74586a2bb6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:36 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:50762
I20260810 14:37:36.301280 23894 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc3ead0b04e43b79d92c74586a2bb6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:36 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:42796
I20260810 14:37:36.304555 24336 tablet_replica.cc:330] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591: stopping tablet replica
I20260810 14:37:36.304869 24336 raft_consensus.cc:2229] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:36.307646 24336 raft_consensus.cc:2258] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:36.309297 24336 ts_tablet_manager.cc:1905] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:36.312191 24337 tablet_replica.cc:330] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b: stopping tablet replica
I20260810 14:37:36.312438 24337 raft_consensus.cc:2229] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:37:36.313367 24337 raft_consensus.cc:2258] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:36.314954 24337 ts_tablet_manager.cc:1905] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 14:37:36 dist-test-slave-h6n0 krb5kdc[23419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372654, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:37:36.330593 24339 tablet_replica.cc:330] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d: stopping tablet replica
I20260810 14:37:36.330818 24339 raft_consensus.cc:2229] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:36.331133 24339 raft_consensus.cc:2258] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:36.331770 24336 ts_tablet_manager.cc:1918] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:37:36.331934 24336 log.cc:1196] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/4cc3ead0b04e43b79d92c74586a2bb6e
I20260810 14:37:36.332456 24336 ts_tablet_manager.cc:1939] T 4cc3ead0b04e43b79d92c74586a2bb6e P fd42b6727a734237afa13f05656e9591: Deleting consensus metadata
I20260810 14:37:36.333702 23468 catalog_manager.cc:4841] TS fd42b6727a734237afa13f05656e9591 (127.21.55.67:44377): tablet 4cc3ead0b04e43b79d92c74586a2bb6e (table test-table [id=99944a1f71924807822b9c1d0cbfdd9f]) successfully deleted
I20260810 14:37:36.335147 24337 ts_tablet_manager.cc:1918] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:37:36.335969 24339 ts_tablet_manager.cc:1905] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:36.340270 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.311074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50784 (local address 127.21.55.65:37417)
0810 14:37:36.311349 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:36.311357 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:36.311388 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:37:36.311459 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:36.311467 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:36.311558 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:36.311799 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:36.314251 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.315398 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.317338 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.318044 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.318147 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.331906 (+ 13759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:36.331922 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:36.331942 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:36.331991 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:36.336269 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:36.337198 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:36.337208 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:36.337212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:36.337372 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:36.337774 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:36.337780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:36.337783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:36.338198 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:36.338515 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:36.339682 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0810 14:37:36.339908 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 14:37:36.335309 24337 log.cc:1196] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/4cc3ead0b04e43b79d92c74586a2bb6e
W20260810 14:37:36.341105 23783 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50784
I20260810 14:37:36.341535 24337 ts_tablet_manager.cc:1939] T 4cc3ead0b04e43b79d92c74586a2bb6e P c6f95c371d9a4f9990154f0c9535202b: Deleting consensus metadata
I20260810 14:37:36.343272 23468 catalog_manager.cc:4841] TS c6f95c371d9a4f9990154f0c9535202b (127.21.55.65:37417): tablet 4cc3ead0b04e43b79d92c74586a2bb6e (table test-table [id=99944a1f71924807822b9c1d0cbfdd9f]) successfully deleted
I20260810 14:37:36.358610 24339 ts_tablet_manager.cc:1918] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:37:36.358796 24339 log.cc:1196] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/4cc3ead0b04e43b79d92c74586a2bb6e
I20260810 14:37:36.359335 24339 ts_tablet_manager.cc:1939] T 4cc3ead0b04e43b79d92c74586a2bb6e P 9a901aa5bf1248c5972e6c1a7638700d: Deleting consensus metadata
I20260810 14:37:36.363190 23466 catalog_manager.cc:4841] TS 9a901aa5bf1248c5972e6c1a7638700d (127.21.55.66:46853): tablet 4cc3ead0b04e43b79d92c74586a2bb6e (table test-table [id=99944a1f71924807822b9c1d0cbfdd9f]) successfully deleted
I20260810 14:37:36.380729 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.362660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.21.55.126:46841)
0810 14:37:36.363310 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:36.363317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:36.364224 (+ 907us) server_negotiation.cc:407] Connection header received
0810 14:37:36.364369 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:36.364374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:36.364480 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:36.364595 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:36.366653 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.367838 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.369681 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.370294 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.370412 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.373588 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:36.373595 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:36.373602 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:36.373643 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:36.377359 (+ 3716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:36.378212 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:36.378218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:36.378222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:36.378390 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:36.378877 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:36.378882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:36.378885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:36.379258 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:36.379659 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:36.380024 (+ 365us) server_negotiation.cc:299] Negotiation successful
0810 14:37:36.380289 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":305,"threads_started":1}
W20260810 14:37:36.381387 23485 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48168
I20260810 14:37:36.385592 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23592
I20260810 14:37:36.405938 24317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.386584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50798 (local address 127.21.55.65:37417)
0810 14:37:36.386789 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:36.386795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:36.387305 (+ 510us) server_negotiation.cc:407] Connection header received
0810 14:37:36.387448 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:36.387454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:36.387553 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:36.387677 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:36.389201 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.390331 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.392294 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.393050 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.393171 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.397025 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:36.397034 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:36.397040 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:36.397143 (+ 103us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:36.401427 (+ 4284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:36.402867 (+ 1440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:36.402873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:36.402877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:36.403072 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:36.403579 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:36.403592 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:36.403597 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:36.403986 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:36.404340 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:36.404942 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 14:37:36.405225 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 14:37:36.406591 23783 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50798
W20260810 14:37:36.407240 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23592: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:37:36.407454 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23592
I20260810 14:37:36.449262 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23794
Aug 10 14:37:36 dist-test-slave-h6n0 krb5kdc[23419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372654, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:37:36.469715 24299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.450095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42818 (local address 127.21.55.66:46853)
0810 14:37:36.450282 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:36.450286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:36.450371 (+ 85us) server_negotiation.cc:407] Connection header received
0810 14:37:36.450610 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:36.450614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:36.450700 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:36.450782 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:36.452468 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.453308 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.455301 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.455885 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.456005 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.462014 (+ 6009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:36.462041 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:36.462062 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:36.462121 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:36.466624 (+ 4503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:36.467425 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:36.467440 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:36.467443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:36.467598 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:36.468013 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:36.468019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:36.468022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:36.468437 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:36.468777 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:36.469081 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 14:37:36.469330 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 14:37:36.470424 23978 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42818
W20260810 14:37:36.471223 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23794: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:37:36.471410 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23794
I20260810 14:37:36.513090 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23989
Aug 10 14:37:36 dist-test-slave-h6n0 krb5kdc[23419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372654, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:37:36.534230 24348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:36.514298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57976 (local address 127.21.55.67:44377)
0810 14:37:36.515038 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:36.515044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:36.515063 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:37:36.515135 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:36.515139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:36.515211 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:36.515328 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:36.517293 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.518114 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.520033 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:36.520477 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:36.520587 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:36.525788 (+ 5201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:36.525806 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:36.525825 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:36.525876 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:36.530006 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:36.531354 (+ 1348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:36.531363 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:36.531366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:36.531694 (+ 328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:36.532262 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:36.532266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:36.532268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:36.532717 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:36.533010 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:36.533513 (+ 503us) server_negotiation.cc:299] Negotiation successful
0810 14:37:36.533810 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":467,"threads_started":1}
W20260810 14:37:36.535319 24189 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57976
W20260810 14:37:36.536245 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23989: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:37:36.536460 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23989
I20260810 14:37:36.569931 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23437
I20260810 14:37:36.571255 23544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:36.694772 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23437
2026-08-10T14:37:36Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5896 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:37:36 dist-test-slave-h6n0 krb5kdc[24354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:37:36 dist-test-slave-h6n0 krb5kdc[24354](info): set up 2 sockets
Aug 10 14:37:36 dist-test-slave-h6n0 krb5kdc[24354](info): commencing operation
krb5kdc: starting...
W20260810 14:37:38.114004 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.367s user 0.006s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:37:38 dist-test-slave-h6n0 krb5kdc[24354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:37:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:37:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:37:38.388988 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38035
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:41345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:38035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:38.772166 24373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:38.772481 24373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:38.772575 24373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:38.779793 24373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:37:38.779918 24373 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 14:37:38.779960 24373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:38.780015 24373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:38.780087 24373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:37:38.780149 24373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:37:38.790084 24373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:38035
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:38.791397 24373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:38.792749 24373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:38.810746 24378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:38.810757 24382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:38.810905 24379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:38.812913 24373 server_base.cc:1034] running on GCE node
I20260810 14:37:38.813899 24373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:38.815762 24373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:38.817529 24373 hybrid_clock.cc:648] HybridClock initialized: now 1786372658817413 us; error 55 us; skew 500 ppm
I20260810 14:37:38.819456 24373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:38 dist-test-slave-h6n0 krb5kdc[24354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:38.831573 24373 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:37:38.834200 24373 webserver.cc:460] Webserver started at http://127.21.55.126:40617/ using document root <none> and password file <none>
I20260810 14:37:38.836153 24373 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:38.836633 24373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:38.837466 24373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:38.841643 24373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "228a9d17d2be45518066a45ae98e4c3d"
format_stamp: "Formatted at 2026-08-10 14:37:38 on dist-test-slave-h6n0"
server_key: "8d56b4a2280a6b1775b12e5408e66c23"
server_key_iv: "fd0b0619f8bc0c48f79301fd02cce02b"
server_key_version: "encryptionkey@0"
I20260810 14:37:38.843322 24373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "228a9d17d2be45518066a45ae98e4c3d"
format_stamp: "Formatted at 2026-08-10 14:37:38 on dist-test-slave-h6n0"
server_key: "8d56b4a2280a6b1775b12e5408e66c23"
server_key_iv: "fd0b0619f8bc0c48f79301fd02cce02b"
server_key_version: "encryptionkey@0"
I20260810 14:37:38.850008 24373 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 14:37:38.860925 24398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:38.863582 24373 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.007s sys 0.004s
I20260810 14:37:38.864307 24373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "228a9d17d2be45518066a45ae98e4c3d"
format_stamp: "Formatted at 2026-08-10 14:37:38 on dist-test-slave-h6n0"
server_key: "8d56b4a2280a6b1775b12e5408e66c23"
server_key_iv: "fd0b0619f8bc0c48f79301fd02cce02b"
server_key_version: "encryptionkey@0"
I20260810 14:37:38.864697 24373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:38.901455 24373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:38.908169 24373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:38.908536 24373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:38.939034 24373 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:38035
I20260810 14:37:38.939039 24486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:38035 every 8 connection(s)
I20260810 14:37:38.941020 24373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:37:38.946784 24487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:38.951272 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 24373
I20260810 14:37:38.951550 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:37:38.952461 21725 external_mini_cluster.cc:1428] Setting key a77c9e880220413d5f9b047e22cc4609
I20260810 14:37:38.963392 24487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d: Bootstrap starting.
I20260810 14:37:38.967438 24487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:38.968851 24487 log.cc:824] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:37:38 dist-test-slave-h6n0 krb5kdc[24354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:38.972312 24487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d: No bootstrap required, opened a new log
I20260810 14:37:38.980504 24487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228a9d17d2be45518066a45ae98e4c3d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38035 } }
I20260810 14:37:38.980932 24491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:38.955401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.21.55.126:38035)
0810 14:37:38.956631 (+ 1230us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:38.956647 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:38.956699 (+ 52us) server_negotiation.cc:407] Connection header received
0810 14:37:38.958956 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:38.958995 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:38.959651 (+ 656us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:38.960156 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:38.962287 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:38.964235 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:38.966235 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:38.967064 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:38.967213 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:38.972343 (+ 5130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:38.972356 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:38.972380 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:38.972445 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:38.976288 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:38.977140 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:38.977148 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:38.977155 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:38.977347 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:38.977796 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:38.977807 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:38.977816 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:38.978430 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:38.978828 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:38.979326 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 14:37:38.979802 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":593,"threads_started":1}
I20260810 14:37:38.981204 24487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:38.981470 24487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 228a9d17d2be45518066a45ae98e4c3d, State: Initialized, Role: FOLLOWER
I20260810 14:37:38.982393 24487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228a9d17d2be45518066a45ae98e4c3d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38035 } }
I20260810 14:37:38.982880 24487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:37:38.983079 24487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:37:38.983451 24487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:38.986728 24487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228a9d17d2be45518066a45ae98e4c3d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38035 } }
I20260810 14:37:38.987562 24487 leader_election.cc:302] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 228a9d17d2be45518066a45ae98e4c3d; no voters:
I20260810 14:37:38.988399 24487 leader_election.cc:288] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:37:38.988588 24498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:38.990193 24498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [term 1 LEADER]: Becoming Leader. State: Replica: 228a9d17d2be45518066a45ae98e4c3d, State: Running, Role: LEADER
I20260810 14:37:38.990939 24498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228a9d17d2be45518066a45ae98e4c3d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38035 } }
I20260810 14:37:38.991510 24487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:37:38.995832 24499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "228a9d17d2be45518066a45ae98e4c3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228a9d17d2be45518066a45ae98e4c3d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38035 } } }
I20260810 14:37:38.996088 24500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 228a9d17d2be45518066a45ae98e4c3d. Latest consensus state: current_term: 1 leader_uuid: "228a9d17d2be45518066a45ae98e4c3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228a9d17d2be45518066a45ae98e4c3d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38035 } } }
I20260810 14:37:38.996423 24499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:38.996506 24500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:38.998888 24511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:37:39.006347 24511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:37:39.015935 24511 catalog_manager.cc:1347] Generated new cluster ID: bff5878b4125400abbf127eea4424bf0
I20260810 14:37:39.016081 24511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:37:39.035609 24511 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:37:39.037622 24511 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:37:39.046660 24511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d: Generated new TSK 0
I20260810 14:37:39.047770 24511 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:37:39.111723 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38035
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:41345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:39.488831 24530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:39.489378 24530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:39.489543 24530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:39.498183 24530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:39.498303 24530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:39.498459 24530 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 14:37:39.498601 24530 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:37:39.498719 24530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:37:39.512284 24530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.55.126:38035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:39.513948 24530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:39.515412 24530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:39.529875 24540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:39.530197 24541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:39.530220 24544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:39.531930 24530 server_base.cc:1034] running on GCE node
I20260810 14:37:39.532979 24530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:39.534400 24530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:39.535729 24530 hybrid_clock.cc:648] HybridClock initialized: now 1786372659535667 us; error 44 us; skew 500 ppm
I20260810 14:37:39.536206 24530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:39 dist-test-slave-h6n0 krb5kdc[24354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372659, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:39.539884 24530 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:37:39.541641 24530 webserver.cc:460] Webserver started at http://127.21.55.65:46637/ using document root <none> and password file <none>
I20260810 14:37:39.542599 24530 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:39.542783 24530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:39.543219 24530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:39.547556 24530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb"
format_stamp: "Formatted at 2026-08-10 14:37:39 on dist-test-slave-h6n0"
server_key: "4e65c1879f3edd7b9902e4be41090eae"
server_key_iv: "3ada9834e9a80c5b99a3533c03389d80"
server_key_version: "encryptionkey@0"
I20260810 14:37:39.548877 24530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb"
format_stamp: "Formatted at 2026-08-10 14:37:39 on dist-test-slave-h6n0"
server_key: "4e65c1879f3edd7b9902e4be41090eae"
server_key_iv: "3ada9834e9a80c5b99a3533c03389d80"
server_key_version: "encryptionkey@0"
I20260810 14:37:39.554627 24530 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:37:39.558418 24559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:39.560113 24530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 14:37:39.560320 24530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb"
format_stamp: "Formatted at 2026-08-10 14:37:39 on dist-test-slave-h6n0"
server_key: "4e65c1879f3edd7b9902e4be41090eae"
server_key_iv: "3ada9834e9a80c5b99a3533c03389d80"
server_key_version: "encryptionkey@0"
I20260810 14:37:39.560637 24530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:39.601105 24530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:39.607445 24530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:39.607831 24530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:39.611393 24530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:39.611563 24530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:39.611737 24530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:39.611900 24530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:37:39 dist-test-slave-h6n0 krb5kdc[24354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372659, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:39.638779 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:39.612121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:53841 (local address 127.21.55.126:38035)
0810 14:37:39.613012 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:39.613019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:39.614217 (+ 1198us) server_negotiation.cc:407] Connection header received
0810 14:37:39.616572 (+ 2355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:39.616578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:39.616684 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:39.616803 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:39.619131 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:39.620141 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:39.624123 (+ 3982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:39.624604 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:39.624750 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:39.631345 (+ 6595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:39.631352 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:39.631355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:39.631392 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:39.634922 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:39.636250 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:39.636256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:39.636261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:39.636406 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:39.636927 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:39.636932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:39.636936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:39.637220 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:39.637374 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:39.638113 (+ 739us) server_negotiation.cc:299] Negotiation successful
0810 14:37:39.638341 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":605,"threads_started":1}
I20260810 14:37:39.640292 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:39.612940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38035 (local address 127.21.55.65:53841)
0810 14:37:39.613998 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:39.614053 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:39.616267 (+ 2214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:39.617036 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:39.617044 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:39.617882 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:39.618917 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:39.618959 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:39.620385 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:39.620394 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:39.623890 (+ 3496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:39.623912 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:39.624876 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:39.624888 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:39.625538 (+ 650us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:39.626940 (+ 1402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:39.626988 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:39.631001 (+ 4013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:39.635138 (+ 4137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:39.635149 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:39.635175 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:39.636109 (+ 934us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:39.636534 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:39.636539 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:39.636548 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:39.636794 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:39.637352 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:39.637360 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:39.637731 (+ 371us) client_negotiation.cc:769] Sending connection context
0810 14:37:39.637997 (+ 266us) client_negotiation.cc:240] Negotiation successful
0810 14:37:39.638611 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":494,"threads_started":1}
I20260810 14:37:39.664724 24729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:42493 every 8 connection(s)
I20260810 14:37:39.664772 24530 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:42493
I20260810 14:37:39.666917 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:39.648215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:59129 (local address 127.21.55.126:38035)
0810 14:37:39.648449 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:39.648458 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:39.649168 (+ 710us) server_negotiation.cc:407] Connection header received
0810 14:37:39.649398 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:39.649401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:39.649467 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:39.649577 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:39.651336 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:39.652318 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:39.654063 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:39.654494 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:39.654657 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:39.657220 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:39.657231 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:39.657239 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:39.657294 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:39.661740 (+ 4446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:39.663280 (+ 1540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:39.663291 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:39.663299 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:39.663556 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:39.664262 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:39.664272 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:39.664279 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:39.664997 (+ 718us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:39.665213 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:39.666153 (+ 940us) server_negotiation.cc:299] Negotiation successful
0810 14:37:39.666474 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:37:39.667003 24530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:37:39.667492 24679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:39.648196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38035 (local address 127.21.55.65:59129)
0810 14:37:39.649025 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:39.649051 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:39.649254 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:39.649610 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:39.649615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:39.650509 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:39.651187 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:39.651200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:39.652462 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:39.652467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:39.653913 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:39.653926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:39.654734 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:39.654740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:39.654921 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:39.655797 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:39.655818 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:39.656974 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:39.661989 (+ 5015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:39.661995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:39.661999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:39.663045 (+ 1046us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:39.663756 (+ 711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:39.663765 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:39.663771 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:39.664025 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:39.665154 (+ 1129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:39.665158 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:39.665636 (+ 478us) client_negotiation.cc:769] Sending connection context
0810 14:37:39.666011 (+ 375us) client_negotiation.cc:240] Negotiation successful
0810 14:37:39.666770 (+ 759us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":408,"threads_started":1}
I20260810 14:37:39.670023 24731 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38035
I20260810 14:37:39.670356 24731 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:39.671576 24731 heartbeater.cc:511] Master 127.21.55.126:38035 requested a full tablet report, sending...
I20260810 14:37:39.674302 24423 ts_manager.cc:194] Registered new tserver with Master: f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493)
I20260810 14:37:39.676220 24423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:53841
I20260810 14:37:39.677304 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 24530
I20260810 14:37:39.677551 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:37:39.678453 21725 external_mini_cluster.cc:1428] Setting key 644febadb514f751b328ce946b232484
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:37:39.735742 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38035
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:41345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:40.117295 24737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:40.117606 24737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:40.117755 24737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:40.125121 24737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:40.125231 24737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:40.125379 24737 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 14:37:40.125449 24737 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:37:40.125566 24737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:37:40.137338 24737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.55.126:38035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:40.138784 24737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:40.140259 24737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:40.158984 24745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:40.159021 24748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:40.159387 24744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:40.160274 24737 server_base.cc:1034] running on GCE node
I20260810 14:37:40.161108 24737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:40.162369 24737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:40.163662 24737 hybrid_clock.cc:648] HybridClock initialized: now 1786372660163601 us; error 50 us; skew 500 ppm
I20260810 14:37:40.164036 24737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:40 dist-test-slave-h6n0 krb5kdc[24354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:40.167812 24737 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:37:40.170095 24737 webserver.cc:460] Webserver started at http://127.21.55.66:37843/ using document root <none> and password file <none>
I20260810 14:37:40.170977 24737 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:40.171113 24737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:40.171502 24737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:40.174970 24737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "ae0dd8eaeba8415e89f85f19448a29c0"
format_stamp: "Formatted at 2026-08-10 14:37:40 on dist-test-slave-h6n0"
server_key: "0597b368e491879ac0300e4812e101e9"
server_key_iv: "fe7f4c4aa05e9c4492a5de76e37d9b17"
server_key_version: "encryptionkey@0"
I20260810 14:37:40.175753 24737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "ae0dd8eaeba8415e89f85f19448a29c0"
format_stamp: "Formatted at 2026-08-10 14:37:40 on dist-test-slave-h6n0"
server_key: "0597b368e491879ac0300e4812e101e9"
server_key_iv: "fe7f4c4aa05e9c4492a5de76e37d9b17"
server_key_version: "encryptionkey@0"
I20260810 14:37:40.181209 24737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 14:37:40.185087 24758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:40.186502 24737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:37:40.186758 24737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "ae0dd8eaeba8415e89f85f19448a29c0"
format_stamp: "Formatted at 2026-08-10 14:37:40 on dist-test-slave-h6n0"
server_key: "0597b368e491879ac0300e4812e101e9"
server_key_iv: "fe7f4c4aa05e9c4492a5de76e37d9b17"
server_key_version: "encryptionkey@0"
I20260810 14:37:40.186966 24737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:40.216955 24737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:40.222366 24737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:40.223117 24737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:40.227123 24737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:40.227240 24737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:40.227385 24737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:40.227497 24737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:37:40 dist-test-slave-h6n0 krb5kdc[24354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:40.257391 24774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.228974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38035 (local address 127.21.55.66:49363)
0810 14:37:40.230226 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:40.230300 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:40.233156 (+ 2856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:40.234262 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:40.234272 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:40.235136 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:40.236080 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:40.236117 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.238188 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.238197 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:40.241439 (+ 3242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:40.241482 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.243509 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.243516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:40.243749 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:40.244492 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:40.244515 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:40.247673 (+ 3158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:40.251170 (+ 3497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:40.251187 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:40.251212 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:40.252081 (+ 869us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:40.252537 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:40.252545 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:40.252551 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:40.253050 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:40.253681 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:40.253690 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:40.254385 (+ 695us) client_negotiation.cc:769] Sending connection context
0810 14:37:40.255099 (+ 714us) client_negotiation.cc:240] Negotiation successful
0810 14:37:40.255915 (+ 816us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":498,"threads_started":1}
I20260810 14:37:40.258857 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.228298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:49363 (local address 127.21.55.126:38035)
0810 14:37:40.230867 (+ 2569us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:40.230873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:40.230896 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:37:40.233518 (+ 2622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:40.233525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:40.233641 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:40.233779 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:40.236352 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.237821 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.241766 (+ 3945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.242959 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.243117 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:40.247953 (+ 4836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:40.247959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:40.247963 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:40.248021 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:40.250977 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:40.252248 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:40.252254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:40.252258 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:40.252398 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:40.253222 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:40.253227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:40.253231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:40.253518 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:40.253708 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:40.258121 (+ 4413us) server_negotiation.cc:299] Negotiation successful
0810 14:37:40.258386 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2434,"thread_start_us":2299,"threads_started":1}
I20260810 14:37:40.286692 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.265987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:33329 (local address 127.21.55.126:38035)
0810 14:37:40.266407 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:40.266419 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:40.266851 (+ 432us) server_negotiation.cc:407] Connection header received
0810 14:37:40.266998 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:40.267008 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:40.267130 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:40.267285 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:40.268261 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.269811 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.273638 (+ 3827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.275028 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.275477 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:40.277787 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:40.277795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:40.277799 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:40.277853 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:40.282862 (+ 5009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:40.283786 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:40.283795 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:40.283801 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:40.284025 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:40.284669 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:40.284677 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:40.284684 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:40.285249 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:40.285448 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:40.286054 (+ 606us) server_negotiation.cc:299] Negotiation successful
0810 14:37:40.286308 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260810 14:37:40.286849 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.265915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38035 (local address 127.21.55.66:33329)
0810 14:37:40.266751 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:40.266768 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:40.266887 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:40.267291 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:40.267294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:40.267752 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:40.268094 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:40.268103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.270105 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.270118 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:40.273445 (+ 3327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:40.273480 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.275449 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.275459 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:40.275676 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:40.276491 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:40.276510 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:40.277526 (+ 1016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:40.283086 (+ 5560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:40.283095 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:40.283103 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:40.283646 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:40.284227 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:40.284236 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:40.284243 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:40.284454 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:40.285414 (+ 960us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:40.285422 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:40.285752 (+ 330us) client_negotiation.cc:769] Sending connection context
0810 14:37:40.285948 (+ 196us) client_negotiation.cc:240] Negotiation successful
0810 14:37:40.286322 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":477,"threads_started":1}
I20260810 14:37:40.297626 24737 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:41467
I20260810 14:37:40.297669 24931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:41467 every 8 connection(s)
I20260810 14:37:40.298987 24737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:37:40.301520 24933 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38035
I20260810 14:37:40.301891 24933 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:40.302706 24933 heartbeater.cc:511] Master 127.21.55.126:38035 requested a full tablet report, sending...
I20260810 14:37:40.304121 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 24737
I20260810 14:37:40.304286 24423 ts_manager.cc:194] Registered new tserver with Master: ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467)
I20260810 14:37:40.304339 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:37:40.305223 21725 external_mini_cluster.cc:1428] Setting key 2fbd9942cebbadb0ea1a246238cb2bc3
I20260810 14:37:40.305730 24423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:49363
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:37:40.359474 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38035
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:41345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:37:40.680197 24731 heartbeater.cc:503] Master 127.21.55.126:38035 was elected leader, sending a full tablet report...
W20260810 14:37:40.737504 24939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:40.737783 24939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:40.737890 24939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:40.745445 24939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:40.745565 24939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:40.745656 24939 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 14:37:40.745708 24939 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:37:40.745783 24939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:37:40.755960 24939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.55.126:38035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:40.757195 24939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:40.758699 24939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:40.772763 24952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:40.772814 24947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:40.772944 24946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:40.773684 24939 server_base.cc:1034] running on GCE node
I20260810 14:37:40.774291 24939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:40.775316 24939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:40.776561 24939 hybrid_clock.cc:648] HybridClock initialized: now 1786372660776505 us; error 49 us; skew 500 ppm
I20260810 14:37:40.776873 24939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:40 dist-test-slave-h6n0 krb5kdc[24354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:40.780499 24939 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:37:40.782255 24939 webserver.cc:460] Webserver started at http://127.21.55.67:40505/ using document root <none> and password file <none>
I20260810 14:37:40.783099 24939 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:40.783247 24939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:40.783577 24939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:40.786417 24939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "e5af58a15c4f434eb0bdbdcfa601bb33"
format_stamp: "Formatted at 2026-08-10 14:37:40 on dist-test-slave-h6n0"
server_key: "3ac731573ea89fe5c74b3710e6f8efb5"
server_key_iv: "03b8b247fa7fc062d67bc4fccc7b2c74"
server_key_version: "encryptionkey@0"
I20260810 14:37:40.787369 24939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "e5af58a15c4f434eb0bdbdcfa601bb33"
format_stamp: "Formatted at 2026-08-10 14:37:40 on dist-test-slave-h6n0"
server_key: "3ac731573ea89fe5c74b3710e6f8efb5"
server_key_iv: "03b8b247fa7fc062d67bc4fccc7b2c74"
server_key_version: "encryptionkey@0"
I20260810 14:37:40.792939 24939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:37:40.796758 24965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:40.798377 24939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:37:40.798626 24939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "e5af58a15c4f434eb0bdbdcfa601bb33"
format_stamp: "Formatted at 2026-08-10 14:37:40 on dist-test-slave-h6n0"
server_key: "3ac731573ea89fe5c74b3710e6f8efb5"
server_key_iv: "03b8b247fa7fc062d67bc4fccc7b2c74"
server_key_version: "encryptionkey@0"
I20260810 14:37:40.798827 24939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:40.815816 24939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:40.820317 24939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:40.820612 24939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:40.823491 24939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:40.823624 24939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:40.823753 24939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:40.824326 24939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:37:40 dist-test-slave-h6n0 krb5kdc[24354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:40.850497 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.824014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:59467 (local address 127.21.55.126:38035)
0810 14:37:40.824473 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:40.824482 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:40.825492 (+ 1010us) server_negotiation.cc:407] Connection header received
0810 14:37:40.827689 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:40.827695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:40.827791 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:40.827908 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:40.829782 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.831109 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.836165 (+ 5056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.836948 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.837101 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:40.842172 (+ 5071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:40.842183 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:40.842191 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:40.842244 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:40.846497 (+ 4253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:40.847533 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:40.847541 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:40.847546 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:40.847682 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:40.848276 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:40.848281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:40.848284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:40.848675 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:40.848836 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:40.849825 (+ 989us) server_negotiation.cc:299] Negotiation successful
0810 14:37:40.850050 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":218,"threads_started":1}
I20260810 14:37:40.852222 24983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.824467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38035 (local address 127.21.55.67:59467)
0810 14:37:40.825334 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:40.825379 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:40.827410 (+ 2031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:40.828157 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:40.828165 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:40.828691 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:40.829572 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:40.829604 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.831284 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.831291 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:40.835943 (+ 4652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:40.835959 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.837186 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.837192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:40.837560 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:40.838465 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:40.838500 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:40.841900 (+ 3400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:40.846723 (+ 4823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:40.846734 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:40.846754 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:40.847341 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:40.847803 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:40.847807 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:40.847813 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:40.848153 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:40.848918 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:40.848924 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:40.849408 (+ 484us) client_negotiation.cc:769] Sending connection context
0810 14:37:40.849714 (+ 306us) client_negotiation.cc:240] Negotiation successful
0810 14:37:40.850272 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":411,"threads_started":1}
I20260810 14:37:40.878278 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.860385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:45755 (local address 127.21.55.126:38035)
0810 14:37:40.860559 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:40.860564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:40.862579 (+ 2015us) server_negotiation.cc:407] Connection header received
0810 14:37:40.862827 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:40.862834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:40.862914 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:40.863025 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:40.864069 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.864943 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.867551 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.868127 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.868278 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:40.870314 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:40.870325 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:40.870333 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:40.870385 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:40.874815 (+ 4430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:40.875932 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:40.875942 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:40.875947 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:40.876146 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:40.876526 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:40.876534 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:40.876540 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:40.876952 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:40.877136 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:40.877603 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 14:37:40.877852 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 14:37:40.878484 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.860894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38035 (local address 127.21.55.67:45755)
0810 14:37:40.862418 (+ 1524us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:40.862441 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:40.862660 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:40.863063 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:40.863069 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:40.863572 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:40.863915 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:40.863925 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.865135 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.865143 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:40.867367 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:40.867390 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.868376 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.868384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:40.868527 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:40.869294 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:40.869317 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:40.870091 (+ 774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:40.874979 (+ 4888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:40.874987 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:40.874993 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:40.875763 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:40.876269 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:40.876273 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:40.876275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:40.876412 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:40.877109 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:40.877117 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:40.877320 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 14:37:40.877497 (+ 177us) client_negotiation.cc:240] Negotiation successful
0810 14:37:40.877914 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1352,"thread_start_us":529,"threads_started":1}
I20260810 14:37:40.887497 24939 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:42615
I20260810 14:37:40.887537 25154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:42615 every 8 connection(s)
I20260810 14:37:40.888747 24939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:37:40.891448 25155 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38035
I20260810 14:37:40.891731 25155 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:40.892441 25155 heartbeater.cc:511] Master 127.21.55.126:38035 requested a full tablet report, sending...
I20260810 14:37:40.893932 24423 ts_manager.cc:194] Registered new tserver with Master: e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67:42615)
I20260810 14:37:40.894942 24423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:59467
I20260810 14:37:40.895187 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 24939
I20260810 14:37:40.895393 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:37:40.896066 21725 external_mini_cluster.cc:1428] Setting key 10ed1b7d1482b5cfed611d3accd2c59f
I20260810 14:37:40.899845 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:37:40 dist-test-slave-h6n0 krb5kdc[24354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:37:40 dist-test-slave-h6n0 krb5kdc[24354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:40.955444 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.938662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36914 (local address 127.21.55.126:38035)
0810 14:37:40.938855 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:40.938863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:40.939134 (+ 271us) server_negotiation.cc:407] Connection header received
0810 14:37:40.939336 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:40.939340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:40.939411 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:40.939502 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:40.941323 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.942072 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.944016 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:40.944455 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:40.944574 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:40.949531 (+ 4957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:40.949536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:40.949540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:40.949574 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:40.952790 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:40.953566 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:40.953569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:40.953572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:40.953697 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:40.954073 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:40.954077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:40.954079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:40.954342 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:40.954645 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:40.954895 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 14:37:40.955117 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:37:40.960421 24423 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:37:40.967834 24423 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:37:41.005229 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.982280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37408 (local address 127.21.55.67:42615)
0810 14:37:40.990099 (+ 7819us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:40.990112 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:40.995114 (+ 5002us) server_negotiation.cc:407] Connection header received
0810 14:37:40.996525 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:40.996538 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:40.996909 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:40.997229 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:40.998407 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.000082 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.002494 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.004024 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.004171 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.004431 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.004693 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.004863 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7635,"thread_start_us":7447,"threads_started":1}
I20260810 14:37:41.005273 25180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.981479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:42615 (local address 127.0.0.1:37408)
0810 14:37:40.994930 (+ 13451us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:40.994957 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:40.996263 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:40.997724 (+ 1461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:40.997736 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:40.997780 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:40.998175 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:40.998186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.000233 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.000240 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.002266 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.002283 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.004225 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.004232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.004441 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.004502 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 14:37:41.004638 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 14:37:41.004844 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13264,"mutex_wait_us":44,"thread_start_us":15323,"threads_started":1}
I20260810 14:37:41.006754 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.980955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42493 (local address 127.0.0.1:50840)
0810 14:37:40.981784 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:40.981828 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:40.981984 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:40.998467 (+ 16483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:40.998473 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:40.998503 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:40.999130 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:40.999144 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.001383 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.001388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.003863 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.003878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.005890 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.005896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.006055 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.006085 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 14:37:41.006180 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 14:37:41.006312 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":113,"thread_start_us":460,"threads_started":1}
I20260810 14:37:41.007249 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.982163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:41467 (local address 127.0.0.1:56338)
0810 14:37:40.996360 (+ 14197us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:40.996399 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:40.996665 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:40.998840 (+ 2175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:40.998852 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:40.998881 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:40.999422 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:40.999435 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.001549 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.001554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.004110 (+ 2556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.004129 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.006236 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.006246 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.006440 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.006480 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 14:37:41.006571 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 14:37:41.006703 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13914,"mutex_wait_us":27,"thread_start_us":12997,"threads_started":1}
I20260810 14:37:41.007400 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.981187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50840 (local address 127.21.55.65:42493)
0810 14:37:40.997745 (+ 16558us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:40.997762 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:40.997806 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:37:40.997935 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:40.997945 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:40.998178 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:40.999380 (+ 1202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:40.999657 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.001225 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.004042 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.005625 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.005815 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.006019 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.006230 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.006399 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16255,"thread_start_us":16076,"threads_started":1}
I20260810 14:37:41.008324 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:40.982612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.21.55.66:41467)
0810 14:37:40.998155 (+ 15543us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:40.998164 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:40.998195 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:37:40.998303 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:40.998311 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:40.998521 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:40.999651 (+ 1130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:40.999899 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.001334 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.004342 (+ 3008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.005986 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.007587 (+ 1601us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.007788 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.007854 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.008037 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15356,"thread_start_us":15202,"threads_started":1}
I20260810 14:37:41.009215 25046 tablet_service.cc:1467] Processing CreateTablet for tablet 7a25750698564d618b1ef40e19b9722c (DEFAULT_TABLE table=test-table [id=32547247a3b04f7eabdcd8bd6ee85104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:41.010648 25046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a25750698564d618b1ef40e19b9722c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:41.011112 24627 tablet_service.cc:1467] Processing CreateTablet for tablet 7a25750698564d618b1ef40e19b9722c (DEFAULT_TABLE table=test-table [id=32547247a3b04f7eabdcd8bd6ee85104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:41.012307 24627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a25750698564d618b1ef40e19b9722c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:41.012599 24830 tablet_service.cc:1467] Processing CreateTablet for tablet 7a25750698564d618b1ef40e19b9722c (DEFAULT_TABLE table=test-table [id=32547247a3b04f7eabdcd8bd6ee85104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:41.013903 24830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a25750698564d618b1ef40e19b9722c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:41.025381 25184 tablet_bootstrap.cc:492] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: Bootstrap starting.
I20260810 14:37:41.027889 25185 tablet_bootstrap.cc:492] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: Bootstrap starting.
I20260810 14:37:41.029645 25184 tablet_bootstrap.cc:654] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:41.031399 25184 log.cc:824] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:41.031742 25186 tablet_bootstrap.cc:492] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: Bootstrap starting.
I20260810 14:37:41.031898 25185 tablet_bootstrap.cc:654] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:41.034466 25185 log.cc:824] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:41.034829 25186 tablet_bootstrap.cc:654] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:41.036036 25186 log.cc:824] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:41.038997 25185 tablet_bootstrap.cc:492] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: No bootstrap required, opened a new log
I20260810 14:37:41.039280 25185 ts_tablet_manager.cc:1397] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20260810 14:37:41.040371 25186 tablet_bootstrap.cc:492] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: No bootstrap required, opened a new log
I20260810 14:37:41.040525 25184 tablet_bootstrap.cc:492] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: No bootstrap required, opened a new log
I20260810 14:37:41.040817 25184 ts_tablet_manager.cc:1397] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.004s
I20260810 14:37:41.040808 25186 ts_tablet_manager.cc:1397] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.004s
I20260810 14:37:41.049364 25185 raft_consensus.cc:348] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.049970 25185 raft_consensus.cc:374] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:41.050087 25185 raft_consensus.cc:729] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5af58a15c4f434eb0bdbdcfa601bb33, State: Initialized, Role: FOLLOWER
I20260810 14:37:41.050962 25185 consensus_queue.cc:260] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.053339 25184 raft_consensus.cc:348] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.053339 25186 raft_consensus.cc:348] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.053900 25184 raft_consensus.cc:374] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:41.053900 25186 raft_consensus.cc:374] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:41.054013 25186 raft_consensus.cc:729] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0dd8eaeba8415e89f85f19448a29c0, State: Initialized, Role: FOLLOWER
I20260810 14:37:41.054013 25184 raft_consensus.cc:729] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2204262a7fe4cb7a28aaa17cab2fbcb, State: Initialized, Role: FOLLOWER
I20260810 14:37:41.054811 25184 consensus_queue.cc:260] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.055055 25155 heartbeater.cc:503] Master 127.21.55.126:38035 was elected leader, sending a full tablet report...
I20260810 14:37:41.055428 25186 consensus_queue.cc:260] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.056144 25185 ts_tablet_manager.cc:1428] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: Time spent starting tablet: real 0.017s user 0.014s sys 0.003s
I20260810 14:37:41.057430 24933 heartbeater.cc:503] Master 127.21.55.126:38035 was elected leader, sending a full tablet report...
I20260810 14:37:41.058133 25186 ts_tablet_manager.cc:1428] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: Time spent starting tablet: real 0.017s user 0.010s sys 0.008s
I20260810 14:37:41.059815 25184 ts_tablet_manager.cc:1428] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: Time spent starting tablet: real 0.019s user 0.014s sys 0.006s
W20260810 14:37:41.140625 25156 tablet.cc:2369] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:37:41.170801 24734 tablet.cc:2369] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:37:41.249672 25197 raft_consensus.cc:482] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:37:41.250067 25197 raft_consensus.cc:504] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.252074 25197 leader_election.cc:288] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493), e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67:42615)
I20260810 14:37:41.259601 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.252717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:55295 (local address 127.21.55.67:42615)
0810 14:37:41.253068 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:41.253077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:41.253569 (+ 492us) server_negotiation.cc:407] Connection header received
0810 14:37:41.253813 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:41.253817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:41.253877 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:41.253973 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:41.254742 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.255576 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.257545 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.258299 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.258491 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.258624 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.259051 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.259305 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260810 14:37:41.259717 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.252540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:42615 (local address 127.21.55.66:55295)
0810 14:37:41.253437 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:41.253454 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:41.253657 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:41.253990 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:41.253994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:41.254090 (+ 96us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:41.254575 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.254593 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.255710 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.255717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.257367 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.257381 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.258553 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.258560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.258738 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.258763 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 14:37:41.259009 (+ 246us) client_negotiation.cc:240] Negotiation successful
0810 14:37:41.259296 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":528,"threads_started":1}
I20260810 14:37:41.260424 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.252553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42493 (local address 127.21.55.66:38829)
0810 14:37:41.253587 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:41.253601 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:41.253756 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:41.254450 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:41.254459 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:41.254511 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:41.255066 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.255079 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.256302 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.256308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.257883 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.257896 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.259227 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.259234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.259551 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.259584 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:37:41.259914 (+ 330us) client_negotiation.cc:240] Negotiation successful
0810 14:37:41.260068 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":1,"thread_start_us":740,"threads_started":1}
I20260810 14:37:41.260454 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.252575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:38829 (local address 127.21.55.65:42493)
0810 14:37:41.252832 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:41.252842 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:41.253692 (+ 850us) server_negotiation.cc:407] Connection header received
0810 14:37:41.253891 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:41.253896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:41.253965 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:41.254359 (+ 394us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:41.255257 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.256128 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.258068 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.258976 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.259177 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.259262 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.259990 (+ 728us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.260210 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 14:37:41.260902 25077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a25750698564d618b1ef40e19b9722c" candidate_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" is_pre_election: true
I20260810 14:37:41.261502 25077 raft_consensus.cc:2454] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae0dd8eaeba8415e89f85f19448a29c0 in term 0.
I20260810 14:37:41.261489 24656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a25750698564d618b1ef40e19b9722c" candidate_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" is_pre_election: true
I20260810 14:37:41.261943 24656 raft_consensus.cc:2454] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae0dd8eaeba8415e89f85f19448a29c0 in term 0.
I20260810 14:37:41.262310 24763 leader_election.cc:302] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0dd8eaeba8415e89f85f19448a29c0, e5af58a15c4f434eb0bdbdcfa601bb33; no voters:
I20260810 14:37:41.262835 25197 raft_consensus.cc:2781] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:37:41.262979 25197 raft_consensus.cc:482] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:37:41.263106 25197 raft_consensus.cc:3037] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:41.265529 25197 raft_consensus.cc:504] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.266223 25197 leader_election.cc:288] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [CANDIDATE]: Term 1 election: Requested vote from peers f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493), e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67:42615)
I20260810 14:37:41.266628 24656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a25750698564d618b1ef40e19b9722c" candidate_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb"
I20260810 14:37:41.266772 25077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a25750698564d618b1ef40e19b9722c" candidate_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33"
I20260810 14:37:41.266848 24656 raft_consensus.cc:3037] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:41.266983 25077 raft_consensus.cc:3037] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:41.270196 24656 raft_consensus.cc:2454] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0dd8eaeba8415e89f85f19448a29c0 in term 1.
I20260810 14:37:41.270196 25077 raft_consensus.cc:2454] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae0dd8eaeba8415e89f85f19448a29c0 in term 1.
I20260810 14:37:41.270601 24763 leader_election.cc:302] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0dd8eaeba8415e89f85f19448a29c0, e5af58a15c4f434eb0bdbdcfa601bb33; no voters:
I20260810 14:37:41.270912 25197 raft_consensus.cc:2781] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:41.271337 25197 raft_consensus.cc:686] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 LEADER]: Becoming Leader. State: Replica: ae0dd8eaeba8415e89f85f19448a29c0, State: Running, Role: LEADER
I20260810 14:37:41.271845 25197 consensus_queue.cc:237] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.276398 24423 catalog_manager.cc:5495] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66). New cstate: current_term: 1 leader_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } health_report { overall_health: UNKNOWN } } }
W20260810 14:37:41.301448 24936 tablet.cc:2369] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 14:37:41 dist-test-slave-h6n0 krb5kdc[24354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:41.343674 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.329082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.21.55.126:38035)
0810 14:37:41.329282 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:41.329287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:41.329855 (+ 568us) server_negotiation.cc:407] Connection header received
0810 14:37:41.330073 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:41.330078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:41.330139 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:41.330241 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:41.331438 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.332178 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.333448 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.333885 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.334017 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.337907 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:41.337912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:41.337916 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:41.337945 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:41.340471 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:41.341189 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:41.341193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:41.341195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:41.341317 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:41.342084 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:41.342088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:41.342090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:41.342359 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:41.342498 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.343241 (+ 743us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.343427 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 14:37:41.344163 25214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.329150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38035 (local address 127.0.0.1:36922)
0810 14:37:41.329751 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:41.329767 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:41.329931 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:41.330264 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:41.330268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:41.331036 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:41.331301 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.331310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.332307 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.332315 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.333311 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.333323 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.334086 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.334093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.334248 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.334952 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:41.334970 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:41.337683 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:41.340620 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:41.340630 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:41.340635 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:41.341075 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:41.341584 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:41.341590 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:41.341594 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:41.341888 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:41.342650 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:41.342661 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:41.342923 (+ 262us) client_negotiation.cc:769] Sending connection context
0810 14:37:41.343129 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 14:37:41.343642 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":64,"thread_start_us":337,"threads_started":1}
I20260810 14:37:41.349642 24423 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:36922:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 14:37:41.350683 24423 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:37:41.357949 25046 tablet_service.cc:1467] Processing CreateTablet for tablet fb7e2a3a16ee4044b32c14872de020b7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57b151ae4aa24fecb2a5b44744ad705f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:37:41.357949 24627 tablet_service.cc:1467] Processing CreateTablet for tablet fb7e2a3a16ee4044b32c14872de020b7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57b151ae4aa24fecb2a5b44744ad705f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:37:41.358129 24830 tablet_service.cc:1467] Processing CreateTablet for tablet fb7e2a3a16ee4044b32c14872de020b7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57b151ae4aa24fecb2a5b44744ad705f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:37:41.358973 25046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb7e2a3a16ee4044b32c14872de020b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:41.358984 24627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb7e2a3a16ee4044b32c14872de020b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:41.359356 24830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb7e2a3a16ee4044b32c14872de020b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:41.366446 25186 tablet_bootstrap.cc:492] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0: Bootstrap starting.
I20260810 14:37:41.366824 25184 tablet_bootstrap.cc:492] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb: Bootstrap starting.
I20260810 14:37:41.367048 25185 tablet_bootstrap.cc:492] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: Bootstrap starting.
I20260810 14:37:41.368508 25184 tablet_bootstrap.cc:654] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:41.368537 25186 tablet_bootstrap.cc:654] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:41.368793 25185 tablet_bootstrap.cc:654] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:41.373744 25185 tablet_bootstrap.cc:492] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: No bootstrap required, opened a new log
I20260810 14:37:41.374047 25184 tablet_bootstrap.cc:492] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb: No bootstrap required, opened a new log
I20260810 14:37:41.374209 25185 ts_tablet_manager.cc:1397] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260810 14:37:41.374265 25184 ts_tablet_manager.cc:1397] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260810 14:37:41.375168 25186 tablet_bootstrap.cc:492] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0: No bootstrap required, opened a new log
I20260810 14:37:41.375316 25186 ts_tablet_manager.cc:1397] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260810 14:37:41.375167 25184 raft_consensus.cc:348] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.375423 25184 raft_consensus.cc:374] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:41.375224 25185 raft_consensus.cc:348] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.375535 25185 raft_consensus.cc:374] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:41.375535 25184 raft_consensus.cc:729] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2204262a7fe4cb7a28aaa17cab2fbcb, State: Initialized, Role: FOLLOWER
I20260810 14:37:41.375640 25185 raft_consensus.cc:729] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5af58a15c4f434eb0bdbdcfa601bb33, State: Initialized, Role: FOLLOWER
I20260810 14:37:41.376189 25186 raft_consensus.cc:348] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.376055 25185 consensus_queue.cc:260] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.376433 25186 raft_consensus.cc:374] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:41.376529 25186 raft_consensus.cc:729] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0dd8eaeba8415e89f85f19448a29c0, State: Initialized, Role: FOLLOWER
I20260810 14:37:41.376749 25186 consensus_queue.cc:260] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.377359 25186 ts_tablet_manager.cc:1428] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:37:41.377005 25195 tablet_replica.cc:439] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } } }
I20260810 14:37:41.377614 25195 tablet_replica.cc:442] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:41.378332 25185 ts_tablet_manager.cc:1428] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 14:37:41.378721 25197 tablet_replica.cc:439] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } } }
I20260810 14:37:41.379215 25197 tablet_replica.cc:442] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:41.375849 25184 consensus_queue.cc:260] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.380528 25184 ts_tablet_manager.cc:1428] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260810 14:37:41.380396 25196 tablet_replica.cc:439] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } } }
I20260810 14:37:41.380870 25196 tablet_replica.cc:442] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:41.494710 25195 raft_consensus.cc:482] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:37:41.494936 25195 raft_consensus.cc:504] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.496718 25195 leader_election.cc:288] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493), ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467)
I20260810 14:37:41.503855 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.497304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:51641 (local address 127.21.55.65:42493)
0810 14:37:41.497536 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:41.497542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:41.497617 (+ 75us) server_negotiation.cc:407] Connection header received
0810 14:37:41.497870 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:41.497878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:41.497970 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:41.498081 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:41.498743 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.499700 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.501581 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.502626 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.502792 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.502891 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.503493 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.503600 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 14:37:41.504074 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.496962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42493 (local address 127.21.55.67:51641)
0810 14:37:41.497500 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:41.497532 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:41.497721 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:41.498188 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:41.498193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:41.498224 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:41.498578 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.498592 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.499831 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.499836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.501409 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.501421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.502885 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.502894 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.503237 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.503287 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 14:37:41.503448 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 14:37:41.503606 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":72,"spinlock_wait_cycles":33664,"thread_start_us":276,"threads_started":1}
I20260810 14:37:41.504760 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.498046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:55041 (local address 127.21.55.66:41467)
0810 14:37:41.498251 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:41.498258 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:41.498279 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:37:41.498455 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:41.498462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:41.498578 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:41.498696 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:41.499216 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.500025 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.502027 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.503723 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.503901 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.503984 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.504387 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.504509 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 14:37:41.504774 25221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.497629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:41467 (local address 127.21.55.67:55041)
0810 14:37:41.498195 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:41.498219 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:41.498324 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:41.498716 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:41.498721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:41.498746 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:41.499055 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.499065 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.500204 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.500208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.501720 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.501734 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.503973 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.503981 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.504213 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.504253 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 14:37:41.504354 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 14:37:41.504502 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":317,"threads_started":1}
I20260810 14:37:41.504810 24656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7e2a3a16ee4044b32c14872de020b7" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" is_pre_election: true
I20260810 14:37:41.505134 24656 raft_consensus.cc:2454] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 0.
I20260810 14:37:41.505632 24973 leader_election.cc:302] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5af58a15c4f434eb0bdbdcfa601bb33, f2204262a7fe4cb7a28aaa17cab2fbcb; no voters:
I20260810 14:37:41.506094 25195 raft_consensus.cc:2781] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:37:41.505906 24861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7e2a3a16ee4044b32c14872de020b7" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" is_pre_election: true
I20260810 14:37:41.506234 25195 raft_consensus.cc:482] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:37:41.506340 25195 raft_consensus.cc:3037] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:41.506402 24861 raft_consensus.cc:2454] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 0.
I20260810 14:37:41.508978 25195 raft_consensus.cc:504] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.509605 25195 leader_election.cc:288] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 election: Requested vote from peers f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493), ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467)
I20260810 14:37:41.510303 24656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7e2a3a16ee4044b32c14872de020b7" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb"
I20260810 14:37:41.510492 24656 raft_consensus.cc:3037] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:41.510800 24861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb7e2a3a16ee4044b32c14872de020b7" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0dd8eaeba8415e89f85f19448a29c0"
I20260810 14:37:41.510994 24861 raft_consensus.cc:3037] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:41.512630 24656 raft_consensus.cc:2454] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 1.
I20260810 14:37:41.513019 24973 leader_election.cc:302] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5af58a15c4f434eb0bdbdcfa601bb33, f2204262a7fe4cb7a28aaa17cab2fbcb; no voters:
I20260810 14:37:41.513324 25195 raft_consensus.cc:2781] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:41.513481 24861 raft_consensus.cc:2454] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 1.
I20260810 14:37:41.513957 25195 raft_consensus.cc:686] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 LEADER]: Becoming Leader. State: Replica: e5af58a15c4f434eb0bdbdcfa601bb33, State: Running, Role: LEADER
I20260810 14:37:41.514446 25195 consensus_queue.cc:237] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.518339 25222 tablet_replica.cc:439] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: TxnStatusTablet state changed. Reason: New leader e5af58a15c4f434eb0bdbdcfa601bb33. Latest consensus state: current_term: 1 leader_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } } }
I20260810 14:37:41.518754 25222 tablet_replica.cc:442] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:37:41.519680 25225 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 14:37:41.519579 24421 catalog_manager.cc:5495] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67). New cstate: current_term: 1 leader_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } health_report { overall_health: HEALTHY } } }
I20260810 14:37:41.519968 25225 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 14:37:41.541446 25214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.535232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:42615 (local address 127.0.0.1:37414)
0810 14:37:41.535484 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:41.535509 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:41.535701 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:41.536292 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:41.536298 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:41.536323 (+ 25us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:37:41.536553 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.536565 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.537852 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.537859 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.539124 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.539137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.539887 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.539895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.540038 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.540092 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:37:41.540871 (+ 779us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:37:41.540893 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 14:37:41.540994 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 14:37:41.541144 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 14:37:41.541451 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.535287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.21.55.67:42615)
0810 14:37:41.535476 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:41.535481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:41.535586 (+ 105us) server_negotiation.cc:407] Connection header received
0810 14:37:41.535935 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:41.535939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:41.536109 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:41.536227 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:41.536724 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.537704 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.539281 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.539716 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.539849 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.540273 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:41.540750 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:41.540908 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.541020 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.541106 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:37:41.549846 24861 raft_consensus.cc:1261] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Refusing update from remote peer e5af58a15c4f434eb0bdbdcfa601bb33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:41.550034 24656 raft_consensus.cc:1261] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Refusing update from remote peer e5af58a15c4f434eb0bdbdcfa601bb33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:41.550897 25222 consensus_queue.cc:1035] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:41.551293 25195 consensus_queue.cc:1035] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:41.564079 25196 tablet_replica.cc:439] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb: TxnStatusTablet state changed. Reason: New leader e5af58a15c4f434eb0bdbdcfa601bb33. Latest consensus state: current_term: 1 leader_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } } }
I20260810 14:37:41.564440 25196 tablet_replica.cc:442] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:41.567992 25197 tablet_replica.cc:439] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0: TxnStatusTablet state changed. Reason: New leader e5af58a15c4f434eb0bdbdcfa601bb33. Latest consensus state: current_term: 1 leader_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } } }
I20260810 14:37:41.568351 25197 tablet_replica.cc:442] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:41.577440 25196 tablet_replica.cc:439] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } } }
I20260810 14:37:41.577709 25196 tablet_replica.cc:442] T fb7e2a3a16ee4044b32c14872de020b7 P f2204262a7fe4cb7a28aaa17cab2fbcb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:41.580132 25222 tablet_replica.cc:439] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } } }
I20260810 14:37:41.580412 25222 tablet_replica.cc:442] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:37:41.581809 25195 tablet_replica.cc:439] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } } }
I20260810 14:37:41.582140 25195 tablet_replica.cc:442] T fb7e2a3a16ee4044b32c14872de020b7 P e5af58a15c4f434eb0bdbdcfa601bb33: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:37:41.582409 25238 mvcc.cc:205] Tried to move back new op lower bound from 7316982421698334720 to 7316982421567258624. Current Snapshot: MvccSnapshot[applied={T|T < 7316982421698334720}]
I20260810 14:37:41.587116 25197 tablet_replica.cc:439] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } } }
I20260810 14:37:41.587414 25197 tablet_replica.cc:442] T fb7e2a3a16ee4044b32c14872de020b7 P ae0dd8eaeba8415e89f85f19448a29c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:37:41.603425 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.591220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56348 (local address 127.21.55.66:41467)
0810 14:37:41.591420 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:41.591428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:41.595320 (+ 3892us) server_negotiation.cc:407] Connection header received
0810 14:37:41.595611 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:41.595616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:41.595721 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:41.595895 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:41.596482 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.597682 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.599560 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.600284 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.600450 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.601269 (+ 819us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:41.602282 (+ 1013us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:41.602490 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.602759 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.602881 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:37:41.621788 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.615994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:34105 (local address 127.21.55.67:42615)
0810 14:37:41.616184 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:41.616188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:41.616886 (+ 698us) server_negotiation.cc:407] Connection header received
0810 14:37:41.617122 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:41.617128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:41.617202 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:41.617298 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:41.617768 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.618504 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.619991 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.620462 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.620621 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.621049 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:41.621147 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:41.621242 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.621383 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.621464 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 14:37:41.621870 25256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.616087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:42615 (local address 127.21.55.66:34105)
0810 14:37:41.616773 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:41.616789 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:41.616984 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:41.617310 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:41.617315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:41.617326 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:37:41.617629 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.617637 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.618676 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.618680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.619859 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.619871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.620658 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.620664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.620789 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.620840 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:37:41.621246 (+ 406us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:37:41.621260 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 14:37:41.621348 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 14:37:41.621476 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":379,"threads_started":1}
I20260810 14:37:41.634392 24655 raft_consensus.cc:1261] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Refusing update from remote peer ae0dd8eaeba8415e89f85f19448a29c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:41.634518 25077 raft_consensus.cc:1261] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Refusing update from remote peer ae0dd8eaeba8415e89f85f19448a29c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:41.635401 25197 consensus_queue.cc:1035] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:41.635797 25203 consensus_queue.cc:1035] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:41.647253 25233 mvcc.cc:205] Tried to move back new op lower bound from 7316982422044078080 to 7316982420573343744. Current Snapshot: MvccSnapshot[applied={T|T < 7316982422044078080}]
I20260810 14:37:41.648524 25237 mvcc.cc:205] Tried to move back new op lower bound from 7316982422044078080 to 7316982420573343744. Current Snapshot: MvccSnapshot[applied={T|T < 7316982422044078080}]
I20260810 14:37:41.676688 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.669925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:45585 (local address 127.21.55.66:41467)
0810 14:37:41.670091 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:41.670098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:41.670782 (+ 684us) server_negotiation.cc:407] Connection header received
0810 14:37:41.671019 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:41.671024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:41.671101 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:41.671162 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:41.672195 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.673103 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.674686 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.675330 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.675477 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.675967 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:41.676120 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:41.676221 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:41.676393 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 14:37:41.676476 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 14:37:41.676929 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:41.669911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:41467 (local address 127.21.55.67:45585)
0810 14:37:41.670659 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:41.670687 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:41.670862 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:41.671521 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:41.671529 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:41.671543 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:37:41.672001 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.672015 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.673255 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.673258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.674513 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:41.674567 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:41.675543 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:41.675550 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:41.675706 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:41.675778 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:37:41.676236 (+ 458us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:37:41.676249 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 14:37:41.676366 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 14:37:41.676513 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":385,"threads_started":1}
I20260810 14:37:41.726687 24421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:41.727010 24421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:41.730973 24421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d: Sending DeleteTablet for 3 replicas of tablet 7a25750698564d618b1ef40e19b9722c
I20260810 14:37:41.732234 24627 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a25750698564d618b1ef40e19b9722c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:41 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:50840
I20260810 14:37:41.732616 24830 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a25750698564d618b1ef40e19b9722c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:41 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:56338
I20260810 14:37:41.735612 25046 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a25750698564d618b1ef40e19b9722c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:41 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:37408
I20260810 14:37:41.740769 25269 tablet_replica.cc:330] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: stopping tablet replica
I20260810 14:37:41.740799 24421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 14:37:41.741264 25269 raft_consensus.cc:2229] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:41.741665 25269 raft_consensus.cc:2258] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 14:37:41.741889 24421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:37:41.745688 25269 ts_tablet_manager.cc:1905] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:41.751611 24624 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdf332448ef40d08234f30eba0a7b60 (DEFAULT_TABLE table=test-table [id=3d82a42b7f5f45f68ac7325ee434641f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:41.752331 24624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdf332448ef40d08234f30eba0a7b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:41.755982 24828 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdf332448ef40d08234f30eba0a7b60 (DEFAULT_TABLE table=test-table [id=3d82a42b7f5f45f68ac7325ee434641f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:41.756078 25046 tablet_service.cc:1467] Processing CreateTablet for tablet 1bdf332448ef40d08234f30eba0a7b60 (DEFAULT_TABLE table=test-table [id=3d82a42b7f5f45f68ac7325ee434641f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:41.756584 24828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdf332448ef40d08234f30eba0a7b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:41.760185 25046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bdf332448ef40d08234f30eba0a7b60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:41.763828 25186 tablet_bootstrap.cc:492] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0: Bootstrap starting.
I20260810 14:37:41.766243 25186 tablet_bootstrap.cc:654] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:41.770756 25269 ts_tablet_manager.cc:1918] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:37:41.770957 25269 log.cc:1196] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/7a25750698564d618b1ef40e19b9722c
I20260810 14:37:41.771628 25269 ts_tablet_manager.cc:1939] T 7a25750698564d618b1ef40e19b9722c P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting consensus metadata
I20260810 14:37:41.773563 24401 catalog_manager.cc:4841] TS e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67:42615): tablet 7a25750698564d618b1ef40e19b9722c (table test-table [id=32547247a3b04f7eabdcd8bd6ee85104]) successfully deleted
I20260810 14:37:41.774976 25184 tablet_bootstrap.cc:492] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb: Bootstrap starting.
I20260810 14:37:41.776660 25184 tablet_bootstrap.cc:654] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:41.782881 25185 tablet_bootstrap.cc:492] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33: Bootstrap starting.
I20260810 14:37:41.785130 25185 tablet_bootstrap.cc:654] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:41.792096 25184 tablet_bootstrap.cc:492] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb: No bootstrap required, opened a new log
I20260810 14:37:41.792346 25184 ts_tablet_manager.cc:1397] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.001s
I20260810 14:37:41.793215 25184 raft_consensus.cc:348] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.793488 25184 raft_consensus.cc:374] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:41.793576 25184 raft_consensus.cc:729] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2204262a7fe4cb7a28aaa17cab2fbcb, State: Initialized, Role: FOLLOWER
I20260810 14:37:41.793859 25184 consensus_queue.cc:260] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.794687 25184 ts_tablet_manager.cc:1428] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:37:41.805846 25268 tablet_replica.cc:330] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: stopping tablet replica
I20260810 14:37:41.807034 25185 tablet_bootstrap.cc:492] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33: No bootstrap required, opened a new log
I20260810 14:37:41.807358 25185 ts_tablet_manager.cc:1397] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.006s
I20260810 14:37:41.808502 25185 raft_consensus.cc:348] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.810557 25185 raft_consensus.cc:374] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:41.810734 25185 raft_consensus.cc:729] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5af58a15c4f434eb0bdbdcfa601bb33, State: Initialized, Role: FOLLOWER
I20260810 14:37:41.811189 25185 consensus_queue.cc:260] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.812043 25185 ts_tablet_manager.cc:1428] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260810 14:37:41.813499 25268 raft_consensus.cc:2229] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:37:41.813709 25186 tablet_bootstrap.cc:492] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0: No bootstrap required, opened a new log
I20260810 14:37:41.813889 25186 ts_tablet_manager.cc:1397] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0: Time spent bootstrapping tablet: real 0.050s user 0.006s sys 0.020s
I20260810 14:37:41.814388 25268 raft_consensus.cc:2258] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:41.814720 25186 raft_consensus.cc:348] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.814968 25186 raft_consensus.cc:374] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:41.815047 25186 raft_consensus.cc:729] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0dd8eaeba8415e89f85f19448a29c0, State: Initialized, Role: FOLLOWER
I20260810 14:37:41.815312 25186 consensus_queue.cc:260] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:41.816553 25186 ts_tablet_manager.cc:1428] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:37:41.824862 25268 ts_tablet_manager.cc:1905] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:41.828243 25267 tablet_replica.cc:330] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: stopping tablet replica
I20260810 14:37:41.829037 25267 raft_consensus.cc:2229] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:41.829841 25267 raft_consensus.cc:2258] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:41.832549 25267 ts_tablet_manager.cc:1905] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:41.835983 25268 ts_tablet_manager.cc:1918] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:37:41.836226 25268 log.cc:1196] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/7a25750698564d618b1ef40e19b9722c
I20260810 14:37:41.836862 25268 ts_tablet_manager.cc:1939] T 7a25750698564d618b1ef40e19b9722c P ae0dd8eaeba8415e89f85f19448a29c0: Deleting consensus metadata
I20260810 14:37:41.839210 24402 catalog_manager.cc:4841] TS ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467): tablet 7a25750698564d618b1ef40e19b9722c (table test-table [id=32547247a3b04f7eabdcd8bd6ee85104]) successfully deleted
I20260810 14:37:41.842156 25267 ts_tablet_manager.cc:1918] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:37:41.842361 25267 log.cc:1196] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/7a25750698564d618b1ef40e19b9722c
I20260810 14:37:41.842995 25267 ts_tablet_manager.cc:1939] T 7a25750698564d618b1ef40e19b9722c P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting consensus metadata
I20260810 14:37:41.844856 24402 catalog_manager.cc:4841] TS f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493): tablet 7a25750698564d618b1ef40e19b9722c (table test-table [id=32547247a3b04f7eabdcd8bd6ee85104]) successfully deleted
I20260810 14:37:42.174247 25247 raft_consensus.cc:482] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:37:42.174481 25247 raft_consensus.cc:504] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:42.175254 25247 leader_election.cc:288] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493), ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467)
I20260810 14:37:42.175845 24655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdf332448ef40d08234f30eba0a7b60" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" is_pre_election: true
I20260810 14:37:42.175982 24861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdf332448ef40d08234f30eba0a7b60" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" is_pre_election: true
I20260810 14:37:42.176121 24655 raft_consensus.cc:2454] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 0.
I20260810 14:37:42.176312 24861 raft_consensus.cc:2454] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 0.
I20260810 14:37:42.176625 24973 leader_election.cc:302] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5af58a15c4f434eb0bdbdcfa601bb33, f2204262a7fe4cb7a28aaa17cab2fbcb; no voters:
I20260810 14:37:42.176923 25247 raft_consensus.cc:2781] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:37:42.177055 25247 raft_consensus.cc:482] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:37:42.177146 25247 raft_consensus.cc:3037] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:42.179244 25247 raft_consensus.cc:504] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:42.179771 25247 leader_election.cc:288] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 election: Requested vote from peers f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493), ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467)
I20260810 14:37:42.180197 24655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdf332448ef40d08234f30eba0a7b60" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb"
I20260810 14:37:42.180411 24655 raft_consensus.cc:3037] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:42.180361 24861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bdf332448ef40d08234f30eba0a7b60" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0dd8eaeba8415e89f85f19448a29c0"
I20260810 14:37:42.180585 24861 raft_consensus.cc:3037] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:42.182497 24655 raft_consensus.cc:2454] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 1.
I20260810 14:37:42.182924 24973 leader_election.cc:302] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5af58a15c4f434eb0bdbdcfa601bb33, f2204262a7fe4cb7a28aaa17cab2fbcb; no voters:
I20260810 14:37:42.183197 25247 raft_consensus.cc:2781] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:42.183358 25247 raft_consensus.cc:686] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 LEADER]: Becoming Leader. State: Replica: e5af58a15c4f434eb0bdbdcfa601bb33, State: Running, Role: LEADER
I20260810 14:37:42.183637 25247 consensus_queue.cc:237] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:42.184607 24861 raft_consensus.cc:2454] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 1.
I20260810 14:37:42.186939 24421 catalog_manager.cc:5495] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67). New cstate: current_term: 1 leader_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } health_report { overall_health: HEALTHY } } }
I20260810 14:37:42.264724 25291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:42.258438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37430 (local address 127.21.55.67:42615)
0810 14:37:42.259211 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:42.259218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:42.259236 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:37:42.259323 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:42.259328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:42.259401 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:42.259521 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:42.259977 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:42.260679 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:42.262063 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:42.262912 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:42.263089 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:42.263509 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:42.263657 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:42.263766 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:42.264186 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 14:37:42.264275 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":556,"threads_started":1}
I20260810 14:37:42.282833 25291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:42.275586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:52781 (local address 127.21.55.67:42615)
0810 14:37:42.275772 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:42.275778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:42.276726 (+ 948us) server_negotiation.cc:407] Connection header received
0810 14:37:42.276899 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:42.276903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:42.276972 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:42.277023 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:42.277830 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:42.279334 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:42.280831 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:42.281352 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:42.281488 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:42.282011 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:42.282144 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:42.282233 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:42.282390 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 14:37:42.282468 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 14:37:42.282933 25295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:42.275390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:42615 (local address 127.21.55.67:52781)
0810 14:37:42.276604 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:42.276630 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:42.276766 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:42.277188 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:42.277192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:42.277202 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:37:42.277453 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:42.277467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:42.279547 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:42.279552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:42.280649 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:42.280662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:42.281589 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:42.281594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:42.281750 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:42.281792 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:37:42.282263 (+ 471us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:37:42.282274 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 14:37:42.282362 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 14:37:42.282474 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"thread_start_us":798,"threads_started":1}
I20260810 14:37:42.293077 24861 raft_consensus.cc:1261] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Refusing update from remote peer e5af58a15c4f434eb0bdbdcfa601bb33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:42.293314 24655 raft_consensus.cc:1261] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Refusing update from remote peer e5af58a15c4f434eb0bdbdcfa601bb33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:42.293823 25296 consensus_queue.cc:1035] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:42.294181 25247 consensus_queue.cc:1035] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 14:37:42.317513 25280 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 14:37:43.925702 24421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:43.926071 24421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:43.931223 24421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d: Sending DeleteTablet for 3 replicas of tablet 1bdf332448ef40d08234f30eba0a7b60
I20260810 14:37:43.932185 25046 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdf332448ef40d08234f30eba0a7b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:43 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:37408
I20260810 14:37:43.932210 24627 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdf332448ef40d08234f30eba0a7b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:43 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:50840
I20260810 14:37:43.932711 24830 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bdf332448ef40d08234f30eba0a7b60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:43 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:56338
I20260810 14:37:43.933211 25347 tablet_replica.cc:330] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb: stopping tablet replica
I20260810 14:37:43.933586 25347 raft_consensus.cc:2229] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:43.933979 25347 raft_consensus.cc:2258] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:43.935385 25348 tablet_replica.cc:330] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33: stopping tablet replica
I20260810 14:37:43.935492 25349 tablet_replica.cc:330] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0: stopping tablet replica
I20260810 14:37:43.935824 25348 raft_consensus.cc:2229] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:37:43.935999 25349 raft_consensus.cc:2229] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:43.936362 25349 raft_consensus.cc:2258] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:43.936447 25347 ts_tablet_manager.cc:1905] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:43.936854 25348 raft_consensus.cc:2258] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:43.937769 25349 ts_tablet_manager.cc:1905] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:43.938277 25348 ts_tablet_manager.cc:1905] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:43.943759 25347 ts_tablet_manager.cc:1918] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:37:43.943953 25347 log.cc:1196] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/1bdf332448ef40d08234f30eba0a7b60
I20260810 14:37:43.944453 25347 ts_tablet_manager.cc:1939] T 1bdf332448ef40d08234f30eba0a7b60 P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting consensus metadata
I20260810 14:37:43.945223 25349 ts_tablet_manager.cc:1918] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:37:43.945398 25349 log.cc:1196] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/1bdf332448ef40d08234f30eba0a7b60
I20260810 14:37:43.945744 24402 catalog_manager.cc:4841] TS f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493): tablet 1bdf332448ef40d08234f30eba0a7b60 (table test-table [id=3d82a42b7f5f45f68ac7325ee434641f]) successfully deleted
I20260810 14:37:43.945904 25349 ts_tablet_manager.cc:1939] T 1bdf332448ef40d08234f30eba0a7b60 P ae0dd8eaeba8415e89f85f19448a29c0: Deleting consensus metadata
I20260810 14:37:43.947472 24402 catalog_manager.cc:4841] TS ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467): tablet 1bdf332448ef40d08234f30eba0a7b60 (table test-table [id=3d82a42b7f5f45f68ac7325ee434641f]) successfully deleted
I20260810 14:37:43.952242 25348 ts_tablet_manager.cc:1918] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:37:43.952401 25348 log.cc:1196] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/1bdf332448ef40d08234f30eba0a7b60
I20260810 14:37:43.952806 25348 ts_tablet_manager.cc:1939] T 1bdf332448ef40d08234f30eba0a7b60 P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting consensus metadata
I20260810 14:37:43.953809 24401 catalog_manager.cc:4841] TS e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67:42615): tablet 1bdf332448ef40d08234f30eba0a7b60 (table test-table [id=3d82a42b7f5f45f68ac7325ee434641f]) successfully deleted
Aug 10 14:37:43 dist-test-slave-h6n0 krb5kdc[24354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372663, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:37:44 dist-test-slave-h6n0 krb5kdc[24354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372663, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:44.015578 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:43.992084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36928 (local address 127.21.55.126:38035)
0810 14:37:43.992846 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:43.992851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:43.992881 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:37:43.992964 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:43.992969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:43.993054 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:43.993213 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:43.994685 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:43.995647 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:43.997144 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:43.997638 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:43.997766 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:44.007978 (+ 10212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:44.007987 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:44.007992 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:44.008078 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:44.012078 (+ 4000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:44.012941 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:44.012947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:44.012949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:44.013138 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:44.013698 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:44.013701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:44.013704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:44.014068 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:44.014294 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:44.014782 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 14:37:44.015023 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":411,"threads_started":1}
I20260810 14:37:44.020753 24421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:37:44.021451 24421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:37:44.029189 24627 tablet_service.cc:1467] Processing CreateTablet for tablet b1eb22664cc843ad8e234b130ee12bbc (DEFAULT_TABLE table=test-table [id=0a4f700cabb641ef88b06fe7b92d8f73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:44.029366 24830 tablet_service.cc:1467] Processing CreateTablet for tablet b1eb22664cc843ad8e234b130ee12bbc (DEFAULT_TABLE table=test-table [id=0a4f700cabb641ef88b06fe7b92d8f73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:44.029493 25046 tablet_service.cc:1467] Processing CreateTablet for tablet b1eb22664cc843ad8e234b130ee12bbc (DEFAULT_TABLE table=test-table [id=0a4f700cabb641ef88b06fe7b92d8f73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:44.029716 24627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1eb22664cc843ad8e234b130ee12bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:44.029978 24830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1eb22664cc843ad8e234b130ee12bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:44.029989 25046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1eb22664cc843ad8e234b130ee12bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:44.035884 25359 tablet_bootstrap.cc:492] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb: Bootstrap starting.
I20260810 14:37:44.037323 25359 tablet_bootstrap.cc:654] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:44.039783 25360 tablet_bootstrap.cc:492] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0: Bootstrap starting.
I20260810 14:37:44.040575 25365 tablet_bootstrap.cc:492] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33: Bootstrap starting.
I20260810 14:37:44.040932 25359 tablet_bootstrap.cc:492] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb: No bootstrap required, opened a new log
I20260810 14:37:44.041101 25359 ts_tablet_manager.cc:1397] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260810 14:37:44.041766 25359 raft_consensus.cc:348] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.042063 25359 raft_consensus.cc:374] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:44.042229 25359 raft_consensus.cc:729] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2204262a7fe4cb7a28aaa17cab2fbcb, State: Initialized, Role: FOLLOWER
I20260810 14:37:44.042374 25365 tablet_bootstrap.cc:654] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:44.042568 25359 consensus_queue.cc:260] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.044215 25360 tablet_bootstrap.cc:654] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:44.044960 25359 ts_tablet_manager.cc:1428] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260810 14:37:44.046216 25365 tablet_bootstrap.cc:492] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33: No bootstrap required, opened a new log
I20260810 14:37:44.046412 25365 ts_tablet_manager.cc:1397] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 14:37:44.046736 25360 tablet_bootstrap.cc:492] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0: No bootstrap required, opened a new log
I20260810 14:37:44.046957 25360 ts_tablet_manager.cc:1397] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 14:37:44.047542 25365 raft_consensus.cc:348] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.047870 25365 raft_consensus.cc:374] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:44.047999 25365 raft_consensus.cc:729] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5af58a15c4f434eb0bdbdcfa601bb33, State: Initialized, Role: FOLLOWER
I20260810 14:37:44.047914 25360 raft_consensus.cc:348] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.048235 25360 raft_consensus.cc:374] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:44.048311 25365 consensus_queue.cc:260] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.048341 25360 raft_consensus.cc:729] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0dd8eaeba8415e89f85f19448a29c0, State: Initialized, Role: FOLLOWER
I20260810 14:37:44.049079 25365 ts_tablet_manager.cc:1428] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 14:37:44.049294 25360 consensus_queue.cc:260] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.050374 25360 ts_tablet_manager.cc:1428] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:37:44.094089 25315 raft_consensus.cc:482] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:37:44.094455 25315 raft_consensus.cc:504] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.095575 25315 leader_election.cc:288] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493), ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467)
I20260810 14:37:44.096231 24655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1eb22664cc843ad8e234b130ee12bbc" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" is_pre_election: true
I20260810 14:37:44.096576 24655 raft_consensus.cc:2454] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 0.
I20260810 14:37:44.096488 24861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1eb22664cc843ad8e234b130ee12bbc" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" is_pre_election: true
I20260810 14:37:44.096813 24861 raft_consensus.cc:2454] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 0.
I20260810 14:37:44.097128 24973 leader_election.cc:302] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5af58a15c4f434eb0bdbdcfa601bb33, f2204262a7fe4cb7a28aaa17cab2fbcb; no voters:
I20260810 14:37:44.097445 25315 raft_consensus.cc:2781] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:37:44.097602 25315 raft_consensus.cc:482] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:37:44.097718 25315 raft_consensus.cc:3037] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:44.100283 25315 raft_consensus.cc:504] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.100944 25315 leader_election.cc:288] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 election: Requested vote from peers f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493), ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467)
I20260810 14:37:44.101428 24655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1eb22664cc843ad8e234b130ee12bbc" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb"
I20260810 14:37:44.101680 24655 raft_consensus.cc:3037] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:44.101661 24861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1eb22664cc843ad8e234b130ee12bbc" candidate_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0dd8eaeba8415e89f85f19448a29c0"
I20260810 14:37:44.101956 24861 raft_consensus.cc:3037] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:44.105196 24655 raft_consensus.cc:2454] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 1.
I20260810 14:37:44.105368 24861 raft_consensus.cc:2454] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5af58a15c4f434eb0bdbdcfa601bb33 in term 1.
I20260810 14:37:44.105827 24973 leader_election.cc:302] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5af58a15c4f434eb0bdbdcfa601bb33, f2204262a7fe4cb7a28aaa17cab2fbcb; no voters:
I20260810 14:37:44.106292 25315 raft_consensus.cc:2781] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:44.106609 25315 raft_consensus.cc:686] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 LEADER]: Becoming Leader. State: Replica: e5af58a15c4f434eb0bdbdcfa601bb33, State: Running, Role: LEADER
I20260810 14:37:44.107033 25315 consensus_queue.cc:237] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.110698 24423 catalog_manager.cc:5495] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 reported cstate change: term changed from 0 to 1, leader changed from <none> to e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67). New cstate: current_term: 1 leader_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } health_report { overall_health: HEALTHY } } }
I20260810 14:37:44.155014 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:44.149273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37440 (local address 127.21.55.67:42615)
0810 14:37:44.149996 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:44.150001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:44.150021 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:37:44.150081 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:44.150085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:44.150157 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:44.150278 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:44.150802 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.151650 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.153060 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.153526 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.153659 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:44.154111 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:44.154243 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:44.154342 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:44.154496 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 14:37:44.154634 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":393,"threads_started":1}
I20260810 14:37:44.167577 24861 raft_consensus.cc:1261] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Refusing update from remote peer e5af58a15c4f434eb0bdbdcfa601bb33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:44.167695 24655 raft_consensus.cc:1261] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Refusing update from remote peer e5af58a15c4f434eb0bdbdcfa601bb33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:44.168358 25315 consensus_queue.cc:1035] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:44.168730 25374 consensus_queue.cc:1035] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:44.225878 24421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:44.226125 24421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:44.229024 24421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d: Sending DeleteTablet for 3 replicas of tablet b1eb22664cc843ad8e234b130ee12bbc
I20260810 14:37:44.229938 24627 tablet_service.cc:1514] Processing DeleteTablet for tablet b1eb22664cc843ad8e234b130ee12bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:44 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:50840
I20260810 14:37:44.230012 24830 tablet_service.cc:1514] Processing DeleteTablet for tablet b1eb22664cc843ad8e234b130ee12bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:44 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:56338
I20260810 14:37:44.230011 25045 tablet_service.cc:1514] Processing DeleteTablet for tablet b1eb22664cc843ad8e234b130ee12bbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:44 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:37408
I20260810 14:37:44.230242 25347 tablet_replica.cc:330] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb: stopping tablet replica
I20260810 14:37:44.230315 25348 tablet_replica.cc:330] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33: stopping tablet replica
I20260810 14:37:44.230464 25347 raft_consensus.cc:2229] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:44.230544 25348 raft_consensus.cc:2229] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:37:44.230726 25347 raft_consensus.cc:2258] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:44.230878 25348 raft_consensus.cc:2258] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:44.231027 25349 tablet_replica.cc:330] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0: stopping tablet replica
I20260810 14:37:44.231249 25349 raft_consensus.cc:2229] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:44.231449 25349 raft_consensus.cc:2258] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:44.231876 25347 ts_tablet_manager.cc:1905] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:44.232136 25348 ts_tablet_manager.cc:1905] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:44.232517 25349 ts_tablet_manager.cc:1905] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:44.232461 24421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:37:44.233361 24421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:37:44.238145 25349 ts_tablet_manager.cc:1918] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:37:44.238322 25349 log.cc:1196] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/b1eb22664cc843ad8e234b130ee12bbc
I20260810 14:37:44.239394 25347 ts_tablet_manager.cc:1918] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:37:44.239593 25347 log.cc:1196] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/b1eb22664cc843ad8e234b130ee12bbc
I20260810 14:37:44.239719 25348 ts_tablet_manager.cc:1918] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:37:44.239889 25348 log.cc:1196] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/b1eb22664cc843ad8e234b130ee12bbc
I20260810 14:37:44.240098 25349 ts_tablet_manager.cc:1939] T b1eb22664cc843ad8e234b130ee12bbc P ae0dd8eaeba8415e89f85f19448a29c0: Deleting consensus metadata
I20260810 14:37:44.240339 25348 ts_tablet_manager.cc:1939] T b1eb22664cc843ad8e234b130ee12bbc P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting consensus metadata
I20260810 14:37:44.240909 25347 ts_tablet_manager.cc:1939] T b1eb22664cc843ad8e234b130ee12bbc P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting consensus metadata
I20260810 14:37:44.241384 24402 catalog_manager.cc:4841] TS ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467): tablet b1eb22664cc843ad8e234b130ee12bbc (table test-table [id=0a4f700cabb641ef88b06fe7b92d8f73]) successfully deleted
I20260810 14:37:44.241415 24401 catalog_manager.cc:4841] TS e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67:42615): tablet b1eb22664cc843ad8e234b130ee12bbc (table test-table [id=0a4f700cabb641ef88b06fe7b92d8f73]) successfully deleted
I20260810 14:37:44.242173 24402 catalog_manager.cc:4841] TS f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493): tablet b1eb22664cc843ad8e234b130ee12bbc (table test-table [id=0a4f700cabb641ef88b06fe7b92d8f73]) successfully deleted
I20260810 14:37:44.243149 24830 tablet_service.cc:1467] Processing CreateTablet for tablet 72a4356b6ebe40c09f66f3d396f76ed3 (DEFAULT_TABLE table=test-table [id=2df95f4124b1490c92c0437d8fd6bc0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:44.243149 24627 tablet_service.cc:1467] Processing CreateTablet for tablet 72a4356b6ebe40c09f66f3d396f76ed3 (DEFAULT_TABLE table=test-table [id=2df95f4124b1490c92c0437d8fd6bc0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:44.243559 24830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72a4356b6ebe40c09f66f3d396f76ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:44.243597 24627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72a4356b6ebe40c09f66f3d396f76ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:44.243577 25045 tablet_service.cc:1467] Processing CreateTablet for tablet 72a4356b6ebe40c09f66f3d396f76ed3 (DEFAULT_TABLE table=test-table [id=2df95f4124b1490c92c0437d8fd6bc0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:37:44.244094 25045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72a4356b6ebe40c09f66f3d396f76ed3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:44.249050 25365 tablet_bootstrap.cc:492] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33: Bootstrap starting.
I20260810 14:37:44.250458 25365 tablet_bootstrap.cc:654] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:44.250975 25359 tablet_bootstrap.cc:492] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb: Bootstrap starting.
I20260810 14:37:44.251541 25360 tablet_bootstrap.cc:492] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0: Bootstrap starting.
I20260810 14:37:44.252672 25359 tablet_bootstrap.cc:654] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:44.253206 25360 tablet_bootstrap.cc:654] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:44.253309 25365 tablet_bootstrap.cc:492] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33: No bootstrap required, opened a new log
I20260810 14:37:44.253484 25365 ts_tablet_manager.cc:1397] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 14:37:44.254251 25365 raft_consensus.cc:348] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.254516 25365 raft_consensus.cc:374] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:44.254640 25365 raft_consensus.cc:729] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5af58a15c4f434eb0bdbdcfa601bb33, State: Initialized, Role: FOLLOWER
I20260810 14:37:44.254947 25359 tablet_bootstrap.cc:492] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb: No bootstrap required, opened a new log
I20260810 14:37:44.254894 25365 consensus_queue.cc:260] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.255102 25359 ts_tablet_manager.cc:1397] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260810 14:37:44.255872 25359 raft_consensus.cc:348] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.256474 25359 raft_consensus.cc:374] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:44.256567 25359 raft_consensus.cc:729] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2204262a7fe4cb7a28aaa17cab2fbcb, State: Initialized, Role: FOLLOWER
I20260810 14:37:44.256599 25365 ts_tablet_manager.cc:1428] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 14:37:44.256817 25359 consensus_queue.cc:260] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.257369 25359 ts_tablet_manager.cc:1428] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 14:37:44.258698 25360 tablet_bootstrap.cc:492] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0: No bootstrap required, opened a new log
I20260810 14:37:44.258970 25360 ts_tablet_manager.cc:1397] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 14:37:44.259848 25360 raft_consensus.cc:348] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.260138 25360 raft_consensus.cc:374] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:44.260239 25360 raft_consensus.cc:729] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae0dd8eaeba8415e89f85f19448a29c0, State: Initialized, Role: FOLLOWER
I20260810 14:37:44.260579 25360 consensus_queue.cc:260] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.261304 25360 ts_tablet_manager.cc:1428] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 14:37:44.302142 25367 raft_consensus.cc:482] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:37:44.302721 25367 raft_consensus.cc:504] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.304916 25367 leader_election.cc:288] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467), e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67:42615)
I20260810 14:37:44.320111 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:44.305869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:59591 (local address 127.21.55.67:42615)
0810 14:37:44.306120 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:44.306129 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:44.313375 (+ 7246us) server_negotiation.cc:407] Connection header received
0810 14:37:44.313637 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:44.313645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:44.313729 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:44.313794 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:44.314677 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.315983 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.318114 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.318939 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.319124 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:44.319206 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:44.319628 (+ 422us) server_negotiation.cc:299] Negotiation successful
0810 14:37:44.319788 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 14:37:44.320171 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:44.305425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:42615 (local address 127.21.55.65:59591)
0810 14:37:44.313281 (+ 7856us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:44.313298 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:44.313478 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:44.314004 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:44.314010 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:44.314053 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:44.314435 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:44.314448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.316155 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.316161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:44.317932 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:44.317943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.319137 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.319143 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:44.319360 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:44.319416 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 14:37:44.319563 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 14:37:44.319705 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7725,"mutex_wait_us":4,"thread_start_us":8006,"threads_started":1}
I20260810 14:37:44.320544 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:44.305210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:41467 (local address 127.21.55.65:39625)
0810 14:37:44.312809 (+ 7599us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:44.312831 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:44.313023 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:44.313505 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:44.313513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:44.313556 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:37:44.313932 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:44.313948 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.315441 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.315446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:44.317383 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:44.317397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.319358 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.319363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:44.319566 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:44.319596 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 14:37:44.319671 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 14:37:44.319761 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7462,"mutex_wait_us":30,"spinlock_wait_cycles":6016,"thread_start_us":7439,"threads_started":1}
I20260810 14:37:44.320731 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:44.305441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:39625 (local address 127.21.55.66:41467)
0810 14:37:44.312624 (+ 7183us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:44.312631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:44.313172 (+ 541us) server_negotiation.cc:407] Connection header received
0810 14:37:44.313251 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:44.313258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:44.313350 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:44.314091 (+ 741us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:37:44.314354 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.315296 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.317574 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.319141 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.319373 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:44.319460 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:44.320159 (+ 699us) server_negotiation.cc:299] Negotiation successful
0810 14:37:44.320319 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7021,"thread_start_us":6841,"threads_started":1}
I20260810 14:37:44.321342 24860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a4356b6ebe40c09f66f3d396f76ed3" candidate_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" is_pre_election: true
I20260810 14:37:44.321595 25076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a4356b6ebe40c09f66f3d396f76ed3" candidate_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" is_pre_election: true
I20260810 14:37:44.321676 24860 raft_consensus.cc:2454] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2204262a7fe4cb7a28aaa17cab2fbcb in term 0.
I20260810 14:37:44.321844 25076 raft_consensus.cc:2454] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2204262a7fe4cb7a28aaa17cab2fbcb in term 0.
I20260810 14:37:44.322193 24566 leader_election.cc:302] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae0dd8eaeba8415e89f85f19448a29c0, f2204262a7fe4cb7a28aaa17cab2fbcb; no voters:
I20260810 14:37:44.322634 25367 raft_consensus.cc:2781] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:37:44.322777 25367 raft_consensus.cc:482] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:37:44.322885 25367 raft_consensus.cc:3037] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:44.325045 25367 raft_consensus.cc:504] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.325721 25367 leader_election.cc:288] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [CANDIDATE]: Term 1 election: Requested vote from peers ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467), e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67:42615)
I20260810 14:37:44.326195 24860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a4356b6ebe40c09f66f3d396f76ed3" candidate_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae0dd8eaeba8415e89f85f19448a29c0"
I20260810 14:37:44.326278 25076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72a4356b6ebe40c09f66f3d396f76ed3" candidate_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33"
I20260810 14:37:44.326413 24860 raft_consensus.cc:3037] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:44.326467 25076 raft_consensus.cc:3037] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:44.329562 25076 raft_consensus.cc:2454] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2204262a7fe4cb7a28aaa17cab2fbcb in term 1.
I20260810 14:37:44.329725 24860 raft_consensus.cc:2454] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2204262a7fe4cb7a28aaa17cab2fbcb in term 1.
I20260810 14:37:44.330057 24564 leader_election.cc:302] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e5af58a15c4f434eb0bdbdcfa601bb33, f2204262a7fe4cb7a28aaa17cab2fbcb; no voters:
I20260810 14:37:44.330371 25367 raft_consensus.cc:2781] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:44.330843 25367 raft_consensus.cc:686] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 LEADER]: Becoming Leader. State: Replica: f2204262a7fe4cb7a28aaa17cab2fbcb, State: Running, Role: LEADER
I20260810 14:37:44.331247 25367 consensus_queue.cc:237] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } }
I20260810 14:37:44.334925 24421 catalog_manager.cc:5495] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb reported cstate change: term changed from 0 to 1, leader changed from <none> to f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65). New cstate: current_term: 1 leader_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2204262a7fe4cb7a28aaa17cab2fbcb" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 } health_report { overall_health: UNKNOWN } } }
I20260810 14:37:44.367385 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:44.360795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50842 (local address 127.21.55.65:42493)
0810 14:37:44.361461 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:44.361469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:44.361488 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:37:44.361588 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:44.361591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:44.361660 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:44.361834 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:44.362312 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.363142 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.364521 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.365002 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.365137 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:44.365636 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:44.366258 (+ 622us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:44.366436 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:44.366851 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 14:37:44.366964 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":321,"threads_started":1}
I20260810 14:37:44.387348 25399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:44.380127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:42615 (local address 127.21.55.65:48541)
0810 14:37:44.381026 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:44.381044 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:44.381212 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:44.381660 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:44.381665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:44.381680 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:37:44.382225 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:44.382236 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.383818 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.383822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:44.385169 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:44.385186 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.386017 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.386023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:44.386176 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:44.386240 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:37:44.386813 (+ 573us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:37:44.386823 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:37:44.386902 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 14:37:44.387023 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":467,"threads_started":1}
I20260810 14:37:44.387461 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:44.380308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:48541 (local address 127.21.55.67:42615)
0810 14:37:44.380522 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:44.380527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:44.381188 (+ 661us) server_negotiation.cc:407] Connection header received
0810 14:37:44.381403 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:44.381407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:44.381491 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:44.381589 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:44.382429 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.383629 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.385348 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.385807 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.385921 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:44.386471 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:44.386691 (+ 220us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:44.386792 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:44.387030 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 14:37:44.387121 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 14:37:44.398198 25076 raft_consensus.cc:1261] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Refusing update from remote peer f2204262a7fe4cb7a28aaa17cab2fbcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:44.398233 24860 raft_consensus.cc:1261] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Refusing update from remote peer f2204262a7fe4cb7a28aaa17cab2fbcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:37:44.398962 25390 consensus_queue.cc:1035] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5af58a15c4f434eb0bdbdcfa601bb33" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:44.399353 25367 consensus_queue.cc:1035] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae0dd8eaeba8415e89f85f19448a29c0" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:37:44.430960 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:44.424820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:54487 (local address 127.21.55.65:42493)
0810 14:37:44.425093 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:44.425101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:44.425516 (+ 415us) server_negotiation.cc:407] Connection header received
0810 14:37:44.425624 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:44.425627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:44.425692 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:44.425773 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:37:44.426451 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.427345 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.428973 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.429632 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.429798 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:44.430209 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:37:44.430304 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:37:44.430401 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:44.430572 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 14:37:44.430648 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260810 14:37:44.431058 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:44.424605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42493 (local address 127.21.55.67:54487)
0810 14:37:44.425385 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:44.425408 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:44.425532 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:44.425796 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:44.425799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:44.425808 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:37:44.426189 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:44.426201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.427495 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.427502 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:44.428821 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:44.428833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:44.429839 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:44.429848 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:44.429994 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:44.430038 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:37:44.430404 (+ 366us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:37:44.430418 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 14:37:44.430515 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 14:37:44.430673 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":387,"threads_started":1}
W20260810 14:37:44.432196 25317 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 14:37:46.053797 24421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:46.054050 24421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:37:46.059309 24421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 228a9d17d2be45518066a45ae98e4c3d: Sending DeleteTablet for 3 replicas of tablet 72a4356b6ebe40c09f66f3d396f76ed3
I20260810 14:37:46.060418 24627 tablet_service.cc:1514] Processing DeleteTablet for tablet 72a4356b6ebe40c09f66f3d396f76ed3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:46 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:50840
I20260810 14:37:46.060705 24830 tablet_service.cc:1514] Processing DeleteTablet for tablet 72a4356b6ebe40c09f66f3d396f76ed3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:46 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:56338
I20260810 14:37:46.060700 25045 tablet_service.cc:1514] Processing DeleteTablet for tablet 72a4356b6ebe40c09f66f3d396f76ed3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:37:46 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:37408
I20260810 14:37:46.061862 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24530
I20260810 14:37:46.073035 25441 tablet_replica.cc:330] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb: stopping tablet replica
I20260810 14:37:46.073150 25443 tablet_replica.cc:330] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33: stopping tablet replica
I20260810 14:37:46.073495 25441 raft_consensus.cc:2229] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:37:46.073539 25443 raft_consensus.cc:2229] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:46.074100 25443 raft_consensus.cc:2258] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:46.074237 25441 raft_consensus.cc:2258] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:46.075822 25443 ts_tablet_manager.cc:1905] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:46.076651 25441 ts_tablet_manager.cc:1905] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:46.077214 25442 tablet_replica.cc:330] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0: stopping tablet replica
I20260810 14:37:46.077477 25442 raft_consensus.cc:2229] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:37:46.077772 25442 raft_consensus.cc:2258] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:37:46.078922 25442 ts_tablet_manager.cc:1905] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:37:46.083794 25443 ts_tablet_manager.cc:1918] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:37:46.084053 25443 log.cc:1196] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/72a4356b6ebe40c09f66f3d396f76ed3
I20260810 14:37:46.084197 25441 ts_tablet_manager.cc:1918] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:37:46.084457 25441 log.cc:1196] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/72a4356b6ebe40c09f66f3d396f76ed3
I20260810 14:37:46.084684 25443 ts_tablet_manager.cc:1939] T 72a4356b6ebe40c09f66f3d396f76ed3 P e5af58a15c4f434eb0bdbdcfa601bb33: Deleting consensus metadata
I20260810 14:37:46.085162 25441 ts_tablet_manager.cc:1939] T 72a4356b6ebe40c09f66f3d396f76ed3 P f2204262a7fe4cb7a28aaa17cab2fbcb: Deleting consensus metadata
I20260810 14:37:46.086423 24402 catalog_manager.cc:4841] TS f2204262a7fe4cb7a28aaa17cab2fbcb (127.21.55.65:42493): tablet 72a4356b6ebe40c09f66f3d396f76ed3 (table test-table [id=2df95f4124b1490c92c0437d8fd6bc0c]) successfully deleted
I20260810 14:37:46.086616 25442 ts_tablet_manager.cc:1918] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:37:46.086589 24401 catalog_manager.cc:4841] TS e5af58a15c4f434eb0bdbdcfa601bb33 (127.21.55.67:42615): tablet 72a4356b6ebe40c09f66f3d396f76ed3 (table test-table [id=2df95f4124b1490c92c0437d8fd6bc0c]) successfully deleted
I20260810 14:37:46.086781 25442 log.cc:1196] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/72a4356b6ebe40c09f66f3d396f76ed3
I20260810 14:37:46.087332 25442 ts_tablet_manager.cc:1939] T 72a4356b6ebe40c09f66f3d396f76ed3 P ae0dd8eaeba8415e89f85f19448a29c0: Deleting consensus metadata
I20260810 14:37:46.088436 24402 catalog_manager.cc:4841] TS ae0dd8eaeba8415e89f85f19448a29c0 (127.21.55.66:41467): tablet 72a4356b6ebe40c09f66f3d396f76ed3 (table test-table [id=2df95f4124b1490c92c0437d8fd6bc0c]) successfully deleted
Aug 10 14:37:46 dist-test-slave-h6n0 krb5kdc[24354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372663, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:37:46.096046 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:46.067035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50856 (local address 127.21.55.65:42493)
0810 14:37:46.070568 (+ 3533us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:46.070580 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:46.077630 (+ 7050us) server_negotiation.cc:407] Connection header received
0810 14:37:46.078110 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:46.078116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:46.078210 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:46.078300 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:46.080147 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:46.081224 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:46.083139 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:46.083813 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:46.083944 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:46.089656 (+ 5712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:46.089669 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:46.089685 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:46.089715 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:46.092842 (+ 3127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:46.093516 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:46.093523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:46.093525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:46.093647 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:46.094062 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:46.094066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:46.094068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:46.094438 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:46.094767 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:46.095418 (+ 651us) server_negotiation.cc:299] Negotiation successful
0810 14:37:46.095771 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3254,"thread_start_us":2947,"threads_started":1}
W20260810 14:37:46.097066 24724 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50856
W20260810 14:37:46.097754 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24530: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:37:46.098023 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 24530
I20260810 14:37:46.136993 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24737
Aug 10 14:37:46 dist-test-slave-h6n0 krb5kdc[24354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372663, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:37:46.158757 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:46.138486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56362 (local address 127.21.55.66:41467)
0810 14:37:46.139389 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:46.139394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:46.139566 (+ 172us) server_negotiation.cc:407] Connection header received
0810 14:37:46.139645 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:46.139650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:46.139738 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:46.139865 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:46.141879 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:46.143389 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:46.144955 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:46.145394 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:46.145507 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:46.152234 (+ 6727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:46.152252 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:46.152268 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:46.152312 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:46.155354 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:46.156006 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:46.156012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:46.156014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:46.156137 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:46.156566 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:46.156573 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:46.156576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:46.157193 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:46.157578 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:46.157907 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 14:37:46.158408 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":420,"threads_started":1}
W20260810 14:37:46.159430 24927 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56362
W20260810 14:37:46.160130 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24737: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:37:46.160274 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 24737
I20260810 14:37:46.198751 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24939
Aug 10 14:37:46 dist-test-slave-h6n0 krb5kdc[24354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372663, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:37:46.216126 25452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:46.199622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37452 (local address 127.21.55.67:42615)
0810 14:37:46.200196 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:46.200201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:46.200218 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:37:46.200271 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:46.200275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:46.200334 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:46.200428 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:46.202160 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:46.203074 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:46.204412 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:46.204820 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:46.204914 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:46.209376 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:46.209406 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:46.209424 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:46.209457 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:46.212743 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:46.213671 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:46.213679 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:46.213681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:46.213821 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:46.214277 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:46.214283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:46.214286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:46.214998 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:46.215403 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:46.215632 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 14:37:46.215803 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":296,"threads_started":1}
W20260810 14:37:46.216876 25149 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37452
W20260810 14:37:46.217461 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24939: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:37:46.217641 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 24939
I20260810 14:37:46.259972 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24373
I20260810 14:37:46.261255 24482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:46.387856 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 24373
2026-08-10T14:37:46Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9696 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:37:46 dist-test-slave-h6n0 krb5kdc[25457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:37:46 dist-test-slave-h6n0 krb5kdc[25457](info): set up 2 sockets
Aug 10 14:37:46 dist-test-slave-h6n0 krb5kdc[25457](info): commencing operation
krb5kdc: starting...
W20260810 14:37:47.781296 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:37:47 dist-test-slave-h6n0 krb5kdc[25457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:37:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:37:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:37:48.031514 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:37587
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:37587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:48.425566 25473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:48.426051 25473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:48.426221 25473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:48.435004 25473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:37:48.435199 25473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:48.435314 25473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:48.435424 25473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:37:48.435518 25473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:37:48.447463 25473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:37587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:37587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:48.449435 25473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:48.451220 25473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:48.466936 25486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:48.466903 25490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:48.466876 25488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:48.467856 25473 server_base.cc:1034] running on GCE node
I20260810 14:37:48.468957 25473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:48.470992 25473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:48.472267 25473 hybrid_clock.cc:648] HybridClock initialized: now 1786372668472208 us; error 52 us; skew 500 ppm
I20260810 14:37:48.472723 25473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:48 dist-test-slave-h6n0 krb5kdc[25457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372668, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:48.476822 25473 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:37:48.478929 25473 webserver.cc:460] Webserver started at http://127.21.55.126:36399/ using document root <none> and password file <none>
I20260810 14:37:48.479840 25473 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:48.480005 25473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:48.480401 25473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:48.484460 25473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "94d7cd00d17f4636a91fe984b50080a7"
format_stamp: "Formatted at 2026-08-10 14:37:48 on dist-test-slave-h6n0"
server_key: "b538389021e75477fb107a961d6db71b"
server_key_iv: "549a338008bc438cc58907d4da1d9207"
server_key_version: "encryptionkey@0"
I20260810 14:37:48.485562 25473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "94d7cd00d17f4636a91fe984b50080a7"
format_stamp: "Formatted at 2026-08-10 14:37:48 on dist-test-slave-h6n0"
server_key: "b538389021e75477fb107a961d6db71b"
server_key_iv: "549a338008bc438cc58907d4da1d9207"
server_key_version: "encryptionkey@0"
I20260810 14:37:48.493263 25473 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260810 14:37:48.498396 25510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:48.500708 25473 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260810 14:37:48.501026 25473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "94d7cd00d17f4636a91fe984b50080a7"
format_stamp: "Formatted at 2026-08-10 14:37:48 on dist-test-slave-h6n0"
server_key: "b538389021e75477fb107a961d6db71b"
server_key_iv: "549a338008bc438cc58907d4da1d9207"
server_key_version: "encryptionkey@0"
I20260810 14:37:48.501331 25473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:48.534209 25473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:48.540066 25473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:48.540459 25473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:48.564975 25473 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:37587
I20260810 14:37:48.565039 25585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:37587 every 8 connection(s)
I20260810 14:37:48.566979 25473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:37:48.568534 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 25473
I20260810 14:37:48.568845 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:37:48.569788 21725 external_mini_cluster.cc:1428] Setting key 9f1212ba0bcd7e5dd13a50bc37479d31
I20260810 14:37:48.573098 25586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:37:48.588515 25586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7: Bootstrap starting.
Aug 10 14:37:48 dist-test-slave-h6n0 krb5kdc[25457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:48.592032 25586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:48.593534 25586 log.cc:824] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:48.597640 25586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7: No bootstrap required, opened a new log
I20260810 14:37:48.600503 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:48.572774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50770 (local address 127.21.55.126:37587)
0810 14:37:48.573941 (+ 1167us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:48.573958 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:48.574020 (+ 62us) server_negotiation.cc:407] Connection header received
0810 14:37:48.576307 (+ 2287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:48.576338 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:48.576989 (+ 651us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:48.577482 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:48.579578 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:48.581418 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:48.583368 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:48.584343 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:48.584540 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:48.590092 (+ 5552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:48.590106 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:48.590127 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:48.590230 (+ 103us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:48.594867 (+ 4637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:48.595995 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:48.596007 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:48.596016 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:48.596272 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:48.596834 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:48.596839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:48.596842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:48.597592 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:48.597911 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:48.598519 (+ 608us) server_negotiation.cc:299] Negotiation successful
0810 14:37:48.599063 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":885,"thread_start_us":560,"threads_started":1}
I20260810 14:37:48.606648 25586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7cd00d17f4636a91fe984b50080a7" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37587 } }
I20260810 14:37:48.607069 25586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:48.607159 25586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d7cd00d17f4636a91fe984b50080a7, State: Initialized, Role: FOLLOWER
I20260810 14:37:48.607722 25586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7cd00d17f4636a91fe984b50080a7" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37587 } }
I20260810 14:37:48.607937 25586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:37:48.608047 25586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:37:48.608201 25586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:48.610328 25586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7cd00d17f4636a91fe984b50080a7" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37587 } }
I20260810 14:37:48.610872 25586 leader_election.cc:302] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94d7cd00d17f4636a91fe984b50080a7; no voters:
I20260810 14:37:48.611500 25586 leader_election.cc:288] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:37:48.611724 25594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:48.612617 25594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [term 1 LEADER]: Becoming Leader. State: Replica: 94d7cd00d17f4636a91fe984b50080a7, State: Running, Role: LEADER
I20260810 14:37:48.613260 25594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7cd00d17f4636a91fe984b50080a7" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37587 } }
I20260810 14:37:48.613831 25586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:37:48.617230 25596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94d7cd00d17f4636a91fe984b50080a7. Latest consensus state: current_term: 1 leader_uuid: "94d7cd00d17f4636a91fe984b50080a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7cd00d17f4636a91fe984b50080a7" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37587 } } }
I20260810 14:37:48.617810 25596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:48.617204 25595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94d7cd00d17f4636a91fe984b50080a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d7cd00d17f4636a91fe984b50080a7" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37587 } } }
I20260810 14:37:48.618005 25595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7 [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:48.621315 25600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:37:48.629621 25600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:37:48.640779 25600 catalog_manager.cc:1347] Generated new cluster ID: 78d333f649aa41ec98df30676ae3d021
I20260810 14:37:48.640918 25600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:37:48.653564 25600 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:37:48.655347 25600 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:37:48.667613 25600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94d7cd00d17f4636a91fe984b50080a7: Generated new TSK 0
I20260810 14:37:48.668870 25600 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:37:48.739666 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37587
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:49.092547 25627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:49.092836 25627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:49.092935 25627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:49.101078 25627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:49.101199 25627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:49.101323 25627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:37:49.112243 25627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:49.113584 25627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:49.115139 25627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:49.129679 25633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:49.129895 25634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:49.129814 25639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:49.131603 25627 server_base.cc:1034] running on GCE node
I20260810 14:37:49.132229 25627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:49.133298 25627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:49.134591 25627 hybrid_clock.cc:648] HybridClock initialized: now 1786372669134485 us; error 61 us; skew 500 ppm
I20260810 14:37:49.134889 25627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:49 dist-test-slave-h6n0 krb5kdc[25457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372669, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:49.143352 25627 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:37:49.145151 25627 webserver.cc:460] Webserver started at http://127.21.55.65:43935/ using document root <none> and password file <none>
I20260810 14:37:49.145890 25627 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:49.146039 25627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:49.146385 25627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:49.150009 25627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "bfd0ed79bf2b4a24842477a893b6b25d"
format_stamp: "Formatted at 2026-08-10 14:37:49 on dist-test-slave-h6n0"
server_key: "463744ed23e286f6605bfd76f48e320c"
server_key_iv: "2fbe93b76df9a72627a8baae3f7f126e"
server_key_version: "encryptionkey@0"
I20260810 14:37:49.150873 25627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "bfd0ed79bf2b4a24842477a893b6b25d"
format_stamp: "Formatted at 2026-08-10 14:37:49 on dist-test-slave-h6n0"
server_key: "463744ed23e286f6605bfd76f48e320c"
server_key_iv: "2fbe93b76df9a72627a8baae3f7f126e"
server_key_version: "encryptionkey@0"
I20260810 14:37:49.156652 25627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 14:37:49.160648 25648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:49.162844 25627 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:37:49.163213 25627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "bfd0ed79bf2b4a24842477a893b6b25d"
format_stamp: "Formatted at 2026-08-10 14:37:49 on dist-test-slave-h6n0"
server_key: "463744ed23e286f6605bfd76f48e320c"
server_key_iv: "2fbe93b76df9a72627a8baae3f7f126e"
server_key_version: "encryptionkey@0"
I20260810 14:37:49.163506 25627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:49.177379 25627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:49.182092 25627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:49.182399 25627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:49.183516 25627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:49.185338 25627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:49.185422 25627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:49.185612 25627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:49.185686 25627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:49.229960 25627 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:46413
I20260810 14:37:49.230007 25812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:46413 every 8 connection(s)
I20260810 14:37:49.231307 25627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:37:49.232080 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 25627
I20260810 14:37:49.232403 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:37:49.233405 21725 external_mini_cluster.cc:1428] Setting key 6c1d6ec709c8acdc4a71d75cdea41826
Aug 10 14:37:49 dist-test-slave-h6n0 krb5kdc[25457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372669, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:49.256717 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:49.234215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:43253 (local address 127.21.55.126:37587)
0810 14:37:49.234884 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:49.234893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:49.235934 (+ 1041us) server_negotiation.cc:407] Connection header received
0810 14:37:49.237895 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:49.237905 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:49.238043 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:49.238221 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:49.240621 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:49.241915 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:49.244734 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:49.245326 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:49.245494 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:49.249865 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:49.249871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:49.249875 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:49.249912 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:49.253189 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:49.253993 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:49.253998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:49.254001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:49.254123 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:49.254617 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:49.254621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:49.254623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:49.254908 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:49.255069 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:49.256043 (+ 974us) server_negotiation.cc:299] Negotiation successful
0810 14:37:49.256283 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":348,"threads_started":1}
I20260810 14:37:49.257918 25818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:49.234797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:37587 (local address 127.21.55.65:43253)
0810 14:37:49.235655 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:49.235703 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:49.237598 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:49.238730 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:49.238751 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:49.239352 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:49.240349 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:49.240387 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:49.242196 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:49.242206 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:49.244343 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:49.244373 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:49.245602 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:49.245610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:49.245870 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:49.246801 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:49.246824 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:49.249603 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:49.253353 (+ 3750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:49.253363 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:49.253381 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:49.253841 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:49.254241 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:49.254246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:49.254249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:49.254451 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:49.255046 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:49.255054 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:49.255437 (+ 383us) client_negotiation.cc:769] Sending connection context
0810 14:37:49.255920 (+ 483us) client_negotiation.cc:240] Negotiation successful
0810 14:37:49.256483 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":370,"threads_started":1}
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
I20260810 14:37:49.260628 25814 heartbeater.cc:347] Connected to a master server at 127.21.55.126:37587
I20260810 14:37:49.261005 25814 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:49.261834 25814 heartbeater.cc:511] Master 127.21.55.126:37587 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
I20260810 14:37:49.264375 25536 ts_manager.cc:194] Registered new tserver with Master: bfd0ed79bf2b4a24842477a893b6b25d (127.21.55.65:46413)
I20260810 14:37:49.266448 25536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:43253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:37:49.295773 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37587
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:49.659718 25821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:49.659996 25821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:49.660094 25821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:49.668531 25821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:49.668656 25821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:49.668772 25821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:37:49.680423 25821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:49.681828 25821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:49.683295 25821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:49.697463 25830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:49.697571 25827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:49.697538 25828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:49.699162 25821 server_base.cc:1034] running on GCE node
I20260810 14:37:49.700157 25821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:49.701197 25821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:49.702481 25821 hybrid_clock.cc:648] HybridClock initialized: now 1786372669702449 us; error 39 us; skew 500 ppm
I20260810 14:37:49.703159 25821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:49 dist-test-slave-h6n0 krb5kdc[25457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372669, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:49.707396 25821 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:37:49.709124 25821 webserver.cc:460] Webserver started at http://127.21.55.66:35993/ using document root <none> and password file <none>
I20260810 14:37:49.709824 25821 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:49.709941 25821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:49.710204 25821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:49.713246 25821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "5fef8d01bf444c34b932d853220c9321"
format_stamp: "Formatted at 2026-08-10 14:37:49 on dist-test-slave-h6n0"
server_key: "0f0b9f5d477ee3461526a438f90a7328"
server_key_iv: "e6310b632fefd72cf594f8c1b4bc170b"
server_key_version: "encryptionkey@0"
I20260810 14:37:49.714087 25821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "5fef8d01bf444c34b932d853220c9321"
format_stamp: "Formatted at 2026-08-10 14:37:49 on dist-test-slave-h6n0"
server_key: "0f0b9f5d477ee3461526a438f90a7328"
server_key_iv: "e6310b632fefd72cf594f8c1b4bc170b"
server_key_version: "encryptionkey@0"
I20260810 14:37:49.719877 25821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:37:49.723923 25842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:49.726014 25821 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:37:49.726235 25821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "5fef8d01bf444c34b932d853220c9321"
format_stamp: "Formatted at 2026-08-10 14:37:49 on dist-test-slave-h6n0"
server_key: "0f0b9f5d477ee3461526a438f90a7328"
server_key_iv: "e6310b632fefd72cf594f8c1b4bc170b"
server_key_version: "encryptionkey@0"
I20260810 14:37:49.726467 25821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:49.737876 25821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:49.742758 25821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:49.743101 25821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:49.744288 25821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:49.745820 25821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:49.745908 25821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:49.746053 25821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:49.746112 25821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:49.787773 25821 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:36029
I20260810 14:37:49.787829 26020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:36029 every 8 connection(s)
I20260810 14:37:49.789310 25821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:37:49.794293 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 25821
I20260810 14:37:49.794595 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:37:49.795291 21725 external_mini_cluster.cc:1428] Setting key 2521b5776d54c96c3f0c8e12d3205902
Aug 10 14:37:49 dist-test-slave-h6n0 krb5kdc[25457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372669, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:49.818843 26024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:49.792174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:46485 (local address 127.21.55.126:37587)
0810 14:37:49.792676 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:49.792681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:49.794285 (+ 1604us) server_negotiation.cc:407] Connection header received
0810 14:37:49.797546 (+ 3261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:49.797552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:49.797663 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:49.797802 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:49.800242 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:49.801330 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:49.804180 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:49.804648 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:49.804752 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:49.811435 (+ 6683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:49.811447 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:49.811457 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:49.811527 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:49.815338 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:49.816205 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:49.816215 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:49.816218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:49.816344 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:49.816814 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:49.816821 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:49.816829 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:49.817218 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:49.817371 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:49.818109 (+ 738us) server_negotiation.cc:299] Negotiation successful
0810 14:37:49.818335 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":278,"threads_started":1}
I20260810 14:37:49.819979 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:49.792638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:37587 (local address 127.21.55.66:46485)
0810 14:37:49.793972 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:49.794079 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:49.797227 (+ 3148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:49.798076 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:49.798088 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:49.798999 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:49.799907 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:49.799937 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:49.801517 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:49.801527 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:49.803990 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:49.804017 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:49.805092 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:49.805123 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:49.805693 (+ 570us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:49.806875 (+ 1182us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:49.806919 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:49.811037 (+ 4118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:49.815522 (+ 4485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:49.815537 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:49.815552 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:49.816054 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:49.816471 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:49.816475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:49.816481 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:49.816681 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:49.817351 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:49.817359 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:49.817722 (+ 363us) client_negotiation.cc:769] Sending connection context
0810 14:37:49.817998 (+ 276us) client_negotiation.cc:240] Negotiation successful
0810 14:37:49.818717 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"thread_start_us":398,"threads_started":1}
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
I20260810 14:37:49.822167 26021 heartbeater.cc:347] Connected to a master server at 127.21.55.126:37587
I20260810 14:37:49.822798 26021 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:49.823879 26021 heartbeater.cc:511] Master 127.21.55.126:37587 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
I20260810 14:37:49.825773 25536 ts_manager.cc:194] Registered new tserver with Master: 5fef8d01bf444c34b932d853220c9321 (127.21.55.66:36029)
I20260810 14:37:49.827236 25536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:46485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:37:49.853854 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37587
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:50.193528 26034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:50.193821 26034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:50.193917 26034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:50.201556 26034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:50.201664 26034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:50.201773 26034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:37:50.211736 26034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:37587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:50.213735 26034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:50.215734 26034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:50.230214 26045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:50.230269 26044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:50.231091 26047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:50.231972 26034 server_base.cc:1034] running on GCE node
I20260810 14:37:50.232528 26034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:50.233574 26034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:50.234867 26034 hybrid_clock.cc:648] HybridClock initialized: now 1786372670234815 us; error 42 us; skew 500 ppm
I20260810 14:37:50.235286 26034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:50 dist-test-slave-h6n0 krb5kdc[25457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:50.238649 26034 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:37:50.240134 26034 webserver.cc:460] Webserver started at http://127.21.55.67:39783/ using document root <none> and password file <none>
I20260810 14:37:50.240954 26034 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:50.241113 26034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:50.241425 26034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:50.244474 26034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "92fd2062e48a4dce9a564376f83da022"
format_stamp: "Formatted at 2026-08-10 14:37:50 on dist-test-slave-h6n0"
server_key: "8ae5006a5190ca35afcf75811eb146a3"
server_key_iv: "cbe8daa99bb07846270edb2d87dae223"
server_key_version: "encryptionkey@0"
I20260810 14:37:50.245253 26034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "92fd2062e48a4dce9a564376f83da022"
format_stamp: "Formatted at 2026-08-10 14:37:50 on dist-test-slave-h6n0"
server_key: "8ae5006a5190ca35afcf75811eb146a3"
server_key_iv: "cbe8daa99bb07846270edb2d87dae223"
server_key_version: "encryptionkey@0"
I20260810 14:37:50.250892 26034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:37:50.254360 26057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:50.255900 26034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:37:50.256109 26034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "92fd2062e48a4dce9a564376f83da022"
format_stamp: "Formatted at 2026-08-10 14:37:50 on dist-test-slave-h6n0"
server_key: "8ae5006a5190ca35afcf75811eb146a3"
server_key_iv: "cbe8daa99bb07846270edb2d87dae223"
server_key_version: "encryptionkey@0"
I20260810 14:37:50.256327 26034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:50.269781 25814 heartbeater.cc:503] Master 127.21.55.126:37587 was elected leader, sending a full tablet report...
I20260810 14:37:50.289283 26034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:50.293990 26034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:50.294299 26034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:50.295514 26034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:50.296977 26034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:50.297080 26034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:50.297209 26034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:50.297268 26034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:50.337225 26034 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:44503
I20260810 14:37:50.337270 26239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:44503 every 8 connection(s)
I20260810 14:37:50.338514 26034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:37:50.345505 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 26034
I20260810 14:37:50.345728 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:37:50.346402 21725 external_mini_cluster.cc:1428] Setting key a0cf2a407bbae01f85e55fab349b6c89
Aug 10 14:37:50 dist-test-slave-h6n0 krb5kdc[25457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:50.363248 26248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:50.342008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:58601 (local address 127.21.55.126:37587)
0810 14:37:50.342620 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:50.342627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:50.343555 (+ 928us) server_negotiation.cc:407] Connection header received
0810 14:37:50.346244 (+ 2689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:50.346251 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:50.346368 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:50.346682 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:50.349251 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:50.350293 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:50.351963 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:50.352374 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:50.352480 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:50.356297 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:50.356302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:50.356305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:50.356337 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:50.358871 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:50.359950 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:50.359958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:50.359961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:50.360377 (+ 416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:50.361289 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:50.361303 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:50.361307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:50.361652 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:50.361876 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:50.362773 (+ 897us) server_negotiation.cc:299] Negotiation successful
0810 14:37:50.362960 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":267,"threads_started":1}
I20260810 14:37:50.364241 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:50.342491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:37587 (local address 127.21.55.67:58601)
0810 14:37:50.343386 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:50.343430 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:50.345788 (+ 2358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:50.346950 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:50.346962 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:50.348167 (+ 1205us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:50.349034 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:50.349053 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:50.350450 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:50.350454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:50.351797 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:50.351826 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:50.352587 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:50.352592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:50.352790 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:50.353449 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:50.353479 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:50.355996 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:50.359011 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:50.359018 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:50.359028 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:50.359553 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:50.360589 (+ 1036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:50.360595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:50.360598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:50.361109 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:50.361893 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:50.361902 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:50.362300 (+ 398us) client_negotiation.cc:769] Sending connection context
0810 14:37:50.362630 (+ 330us) client_negotiation.cc:240] Negotiation successful
0810 14:37:50.363096 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":357,"threads_started":1}
I20260810 14:37:50.365839 26244 heartbeater.cc:347] Connected to a master server at 127.21.55.126:37587
I20260810 14:37:50.366173 26244 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:50.366973 26244 heartbeater.cc:511] Master 127.21.55.126:37587 requested a full tablet report, sending...
I20260810 14:37:50.368441 25536 ts_manager.cc:194] Registered new tserver with Master: 92fd2062e48a4dce9a564376f83da022 (127.21.55.67:44503)
I20260810 14:37:50.369320 25536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:58601
I20260810 14:37:50.375150 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 14:37:50.414237 26248 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:37:50.411721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35424 (local address 127.21.55.126:37587)
0810 14:37:50.411988 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:50.411998 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:50.412328 (+ 330us) server_negotiation.cc:407] Connection header received
0810 14:37:50.412468 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:50.412472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:50.412556 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:50.412644 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:50.413939 (+ 1295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35424: BlockingRecv error: recv got EOF from 127.0.0.1:35424 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:37:50.418982 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 25627
W20260810 14:37:50.421968 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 25627: Not authorized: Client connection negotiation failed: client connection to 127.21.55.65:46413: server requires authentication, but client does not have Kerberos credentials available
I20260810 14:37:50.422150 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 25627
I20260810 14:37:50.451253 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 25821
W20260810 14:37:50.454753 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 25821: Not authorized: Client connection negotiation failed: client connection to 127.21.55.66:36029: server requires authentication, but client does not have Kerberos credentials available
I20260810 14:37:50.454957 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 25821
W20260810 14:37:50.455009 26264 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:37:50.452574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.21.55.66:36029)
0810 14:37:50.453259 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:50.453267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:50.453296 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:37:50.453380 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:50.453387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:50.453608 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:50.453817 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:50.454683 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42584: BlockingRecv error: recv got EOF from 127.0.0.1:42584 (error 108)
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":320,"threads_started":1}
I20260810 14:37:50.487396 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 26034
W20260810 14:37:50.491452 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 26034: Not authorized: Client connection negotiation failed: client connection to 127.21.55.67:44503: server requires authentication, but client does not have Kerberos credentials available
I20260810 14:37:50.491652 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 26034
I20260810 14:37:50.520308 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 25473
I20260810 14:37:50.521752 25579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:50.632798 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 25473
2026-08-10T14:37:50Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4238 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:37:50 dist-test-slave-h6n0 krb5kdc[26269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:37:50 dist-test-slave-h6n0 krb5kdc[26269](info): set up 2 sockets
Aug 10 14:37:50 dist-test-slave-h6n0 krb5kdc[26269](info): commencing operation
krb5kdc: starting...
W20260810 14:37:51.979573 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:37:52 dist-test-slave-h6n0 krb5kdc[26269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:37:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:37:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:37:52.206694 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:41487
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:39445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:41487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:52.547936 26290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:52.548305 26290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:52.548498 26290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:52.556277 26290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:37:52.556499 26290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:52.556608 26290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:52.556718 26290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:37:52.556813 26290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:37:52.566893 26290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:41487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:41487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:52.568192 26290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:52.569453 26290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:52.583099 26297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:52.583122 26300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:52.583326 26298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:52.585246 26290 server_base.cc:1034] running on GCE node
I20260810 14:37:52.586066 26290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:52.587527 26290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:52.588768 26290 hybrid_clock.cc:648] HybridClock initialized: now 1786372672588735 us; error 51 us; skew 500 ppm
I20260810 14:37:52.589121 26290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:52 dist-test-slave-h6n0 krb5kdc[26269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372672, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:52.592293 26290 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:37:52.593789 26290 webserver.cc:460] Webserver started at http://127.21.55.126:36415/ using document root <none> and password file <none>
I20260810 14:37:52.594426 26290 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:52.594594 26290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:52.594937 26290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:52.597860 26290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "3b613dcb9dcc43599d1e2fb3a6515f2a"
format_stamp: "Formatted at 2026-08-10 14:37:52 on dist-test-slave-h6n0"
server_key: "3ab2b01ad47bf393c7cb7a658b2d6721"
server_key_iv: "456a261dca9e541ff2fd64657020f05c"
server_key_version: "encryptionkey@0"
I20260810 14:37:52.598723 26290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "3b613dcb9dcc43599d1e2fb3a6515f2a"
format_stamp: "Formatted at 2026-08-10 14:37:52 on dist-test-slave-h6n0"
server_key: "3ab2b01ad47bf393c7cb7a658b2d6721"
server_key_iv: "456a261dca9e541ff2fd64657020f05c"
server_key_version: "encryptionkey@0"
I20260810 14:37:52.606360 26290 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.002s
I20260810 14:37:52.610090 26308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:52.611554 26290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:37:52.611743 26290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "3b613dcb9dcc43599d1e2fb3a6515f2a"
format_stamp: "Formatted at 2026-08-10 14:37:52 on dist-test-slave-h6n0"
server_key: "3ab2b01ad47bf393c7cb7a658b2d6721"
server_key_iv: "456a261dca9e541ff2fd64657020f05c"
server_key_version: "encryptionkey@0"
I20260810 14:37:52.611932 26290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:52.638571 26290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:52.643913 26290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:52.644244 26290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:52.665280 26290 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:41487
I20260810 14:37:52.665375 26394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:41487 every 8 connection(s)
I20260810 14:37:52.666647 26290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:37:52.667564 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 26290
I20260810 14:37:52.667903 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:37:52.668783 21725 external_mini_cluster.cc:1428] Setting key 10989a30fe51d9b9ede1504fa1074d0b
I20260810 14:37:52.671032 26395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 14:37:52 dist-test-slave-h6n0 krb5kdc[26269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:52.686704 26395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a: Bootstrap starting.
I20260810 14:37:52.691202 26395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:52.692765 26395 log.cc:824] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:52.695770 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:52.670948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38474 (local address 127.21.55.126:41487)
0810 14:37:52.671855 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:52.671872 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:52.671923 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:37:52.674095 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:52.674122 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:52.674653 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:52.675301 (+ 648us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:52.676502 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:52.679226 (+ 2724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:52.680554 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:52.681750 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:52.681956 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:52.686384 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:52.686396 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:52.686415 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:52.686466 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:52.690772 (+ 4306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:52.691497 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:52.691506 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:52.691514 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:52.691761 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:52.692155 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:52.692161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:52.692164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:52.693120 (+ 956us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:52.693430 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:52.693969 (+ 539us) server_negotiation.cc:299] Negotiation successful
0810 14:37:52.694376 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":395,"threads_started":1}
I20260810 14:37:52.696789 26395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a: No bootstrap required, opened a new log
I20260810 14:37:52.700505 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:37:52.707783 26395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b613dcb9dcc43599d1e2fb3a6515f2a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41487 } }
I20260810 14:37:52.708281 26395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:52.708393 26395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b613dcb9dcc43599d1e2fb3a6515f2a, State: Initialized, Role: FOLLOWER
I20260810 14:37:52.709100 26395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b613dcb9dcc43599d1e2fb3a6515f2a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41487 } }
I20260810 14:37:52.709400 26395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:37:52.709546 26395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:37:52.709724 26395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:52.712018 26395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b613dcb9dcc43599d1e2fb3a6515f2a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41487 } }
I20260810 14:37:52.712503 26395 leader_election.cc:302] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b613dcb9dcc43599d1e2fb3a6515f2a; no voters:
I20260810 14:37:52.713097 26395 leader_election.cc:288] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:37:52.713236 26409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:52.715411 26395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:37:52.720274 26409 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [term 1 LEADER]: Becoming Leader. State: Replica: 3b613dcb9dcc43599d1e2fb3a6515f2a, State: Running, Role: LEADER
I20260810 14:37:52.720885 26409 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b613dcb9dcc43599d1e2fb3a6515f2a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41487 } }
I20260810 14:37:52.723737 26410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b613dcb9dcc43599d1e2fb3a6515f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b613dcb9dcc43599d1e2fb3a6515f2a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41487 } } }
I20260810 14:37:52.724355 26410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:52.724512 26411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b613dcb9dcc43599d1e2fb3a6515f2a. Latest consensus state: current_term: 1 leader_uuid: "3b613dcb9dcc43599d1e2fb3a6515f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b613dcb9dcc43599d1e2fb3a6515f2a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41487 } } }
I20260810 14:37:52.725129 26411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:52.730719 26421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:37:52.745733 26421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:37:52.765198 26421 catalog_manager.cc:1347] Generated new cluster ID: 54aea258071a4dd0b1dfab8016b8cd02
I20260810 14:37:52.765327 26421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:37:52.775839 26450 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 14:37:52.788378 26421 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:37:52.790217 26421 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:37:52.790791 26401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:52.778959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38490 (local address 127.21.55.126:41487)
0810 14:37:52.779199 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:52.779208 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:52.779980 (+ 772us) server_negotiation.cc:407] Connection header received
0810 14:37:52.780210 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:52.780216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:52.780326 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:52.780495 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:52.781993 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:52.783177 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:52.785410 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:52.786065 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:52.786206 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:52.788835 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:52.788840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:52.788844 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:37:52.789660 (+ 816us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:52.789724 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:37:52.789757 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:52.789889 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:52.790011 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 14:37:52.790110 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 14:37:52.812428 26421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b613dcb9dcc43599d1e2fb3a6515f2a: Generated new TSK 0
I20260810 14:37:52.813344 26421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:37:52.831874 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 26290
I20260810 14:37:52.833165 26387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:52.924240 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 26290
2026-08-10T14:37:52Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2285 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:37:53 dist-test-slave-h6n0 krb5kdc[26456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:37:53 dist-test-slave-h6n0 krb5kdc[26456](info): set up 2 sockets
Aug 10 14:37:53 dist-test-slave-h6n0 krb5kdc[26456](info): commencing operation
krb5kdc: starting...
W20260810 14:37:54.246923 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:37:54 dist-test-slave-h6n0 krb5kdc[26456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:37:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:37:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:37:54.467357 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:43917
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:43917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:54.831158 26481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:54.831462 26481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:54.831620 26481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:54.838809 26481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:37:54.838910 26481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:54.838999 26481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:54.839100 26481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:37:54.839187 26481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:37:54.849006 26481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:43917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:43917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:54.850335 26481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:54.852169 26481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:54.864984 26489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:54.865237 26490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:54.865695 26493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:54.867534 26481 server_base.cc:1034] running on GCE node
I20260810 14:37:54.868345 26481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:54.869741 26481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:54.871119 26481 hybrid_clock.cc:648] HybridClock initialized: now 1786372674871046 us; error 63 us; skew 500 ppm
I20260810 14:37:54.871516 26481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:54 dist-test-slave-h6n0 krb5kdc[26456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372674, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:54.874791 26481 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:37:54.876508 26481 webserver.cc:460] Webserver started at http://127.21.55.126:45389/ using document root <none> and password file <none>
I20260810 14:37:54.877194 26481 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:54.877346 26481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:54.877661 26481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:54.881018 26481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "dc794c89d0f84d5c80226925cae69871"
format_stamp: "Formatted at 2026-08-10 14:37:54 on dist-test-slave-h6n0"
server_key: "0e3617b51636b5bdfc5a3e3e4fc72187"
server_key_iv: "93721b16462009add66d35d2df8c9d36"
server_key_version: "encryptionkey@0"
I20260810 14:37:54.881886 26481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "dc794c89d0f84d5c80226925cae69871"
format_stamp: "Formatted at 2026-08-10 14:37:54 on dist-test-slave-h6n0"
server_key: "0e3617b51636b5bdfc5a3e3e4fc72187"
server_key_iv: "93721b16462009add66d35d2df8c9d36"
server_key_version: "encryptionkey@0"
I20260810 14:37:54.887437 26481 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:37:54.891415 26503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:54.893050 26481 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 14:37:54.893262 26481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "dc794c89d0f84d5c80226925cae69871"
format_stamp: "Formatted at 2026-08-10 14:37:54 on dist-test-slave-h6n0"
server_key: "0e3617b51636b5bdfc5a3e3e4fc72187"
server_key_iv: "93721b16462009add66d35d2df8c9d36"
server_key_version: "encryptionkey@0"
I20260810 14:37:54.893481 26481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:54.921373 26481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:54.927029 26481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:54.927377 26481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:54.946413 26481 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:43917
I20260810 14:37:54.946458 26577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:43917 every 8 connection(s)
I20260810 14:37:54.947818 26481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:37:54.949147 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 26481
I20260810 14:37:54.949363 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:37:54.950222 21725 external_mini_cluster.cc:1428] Setting key 241c3d9f3c1c9f97d670141465ed0bad
I20260810 14:37:54.952680 26578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 14:37:54 dist-test-slave-h6n0 krb5kdc[26456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:54.965859 26578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871: Bootstrap starting.
I20260810 14:37:54.969959 26578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:54.971302 26578 log.cc:824] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:54.975018 26578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871: No bootstrap required, opened a new log
I20260810 14:37:54.975893 26583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:54.952421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38996 (local address 127.21.55.126:43917)
0810 14:37:54.953259 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:54.953270 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:54.953309 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:37:54.954977 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:54.954996 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:54.955352 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:54.955671 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:54.957428 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:54.958882 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:54.960711 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:54.961779 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:54.961950 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:54.966816 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:54.966826 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:54.966844 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:54.966897 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:54.970996 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:54.971796 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:54.971808 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:54.971819 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:54.972079 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:54.972531 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:54.972534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:54.972536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:54.973140 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:54.973443 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:54.974102 (+ 659us) server_negotiation.cc:299] Negotiation successful
0810 14:37:54.974549 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":417,"threads_started":1}
I20260810 14:37:54.983399 26578 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc794c89d0f84d5c80226925cae69871" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43917 } }
I20260810 14:37:54.983836 26578 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:54.983939 26578 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc794c89d0f84d5c80226925cae69871, State: Initialized, Role: FOLLOWER
I20260810 14:37:54.984511 26578 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc794c89d0f84d5c80226925cae69871" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43917 } }
I20260810 14:37:54.984737 26578 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:37:54.984887 26578 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:37:54.985069 26578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:54.987262 26578 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc794c89d0f84d5c80226925cae69871" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43917 } }
I20260810 14:37:54.987735 26578 leader_election.cc:302] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc794c89d0f84d5c80226925cae69871; no voters:
I20260810 14:37:54.988461 26578 leader_election.cc:288] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:37:54.988893 26586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:54.989889 26586 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [term 1 LEADER]: Becoming Leader. State: Replica: dc794c89d0f84d5c80226925cae69871, State: Running, Role: LEADER
I20260810 14:37:54.990783 26578 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:37:54.990593 26586 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc794c89d0f84d5c80226925cae69871" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43917 } }
I20260810 14:37:54.993798 26590 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc794c89d0f84d5c80226925cae69871. Latest consensus state: current_term: 1 leader_uuid: "dc794c89d0f84d5c80226925cae69871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc794c89d0f84d5c80226925cae69871" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43917 } } }
I20260810 14:37:54.994731 26590 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:54.994293 26587 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc794c89d0f84d5c80226925cae69871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc794c89d0f84d5c80226925cae69871" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43917 } } }
I20260810 14:37:54.995303 26587 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871 [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:54.996419 26607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:37:55.000813 26607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:37:55.009774 26607 catalog_manager.cc:1347] Generated new cluster ID: d4f61eed7a204816913c3a70ac213eb0
I20260810 14:37:55.009908 26607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:37:55.034482 26607 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:37:55.036258 26607 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:37:55.045972 26607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc794c89d0f84d5c80226925cae69871: Generated new TSK 0
I20260810 14:37:55.047312 26607 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:37:55.110174 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43917
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:55.465165 26622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:55.465442 26622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:55.465620 26622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:55.472771 26622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:55.472875 26622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:55.473055 26622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:37:55.483570 26622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:55.484911 26622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:55.486275 26622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:55.500100 26634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:55.500100 26635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:55.500137 26639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:55.501411 26622 server_base.cc:1034] running on GCE node
I20260810 14:37:55.502054 26622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:55.503042 26622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:55.504251 26622 hybrid_clock.cc:648] HybridClock initialized: now 1786372675504214 us; error 36 us; skew 500 ppm
I20260810 14:37:55.504585 26622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:55 dist-test-slave-h6n0 krb5kdc[26456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372675, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:55.507992 26622 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:37:55.509719 26622 webserver.cc:460] Webserver started at http://127.21.55.65:45999/ using document root <none> and password file <none>
I20260810 14:37:55.510404 26622 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:55.510608 26622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:55.510922 26622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:55.513866 26622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "5e02a00bd88444a58924d721d1db857f"
format_stamp: "Formatted at 2026-08-10 14:37:55 on dist-test-slave-h6n0"
server_key: "fca903eaf338e9f8d51477ccdfe38636"
server_key_iv: "2c9ddd6313392afbeb9d7f05648c9e3b"
server_key_version: "encryptionkey@0"
I20260810 14:37:55.514739 26622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "5e02a00bd88444a58924d721d1db857f"
format_stamp: "Formatted at 2026-08-10 14:37:55 on dist-test-slave-h6n0"
server_key: "fca903eaf338e9f8d51477ccdfe38636"
server_key_iv: "2c9ddd6313392afbeb9d7f05648c9e3b"
server_key_version: "encryptionkey@0"
I20260810 14:37:55.520440 26622 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 14:37:55.524595 26647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:55.526677 26622 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 14:37:55.526881 26622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "5e02a00bd88444a58924d721d1db857f"
format_stamp: "Formatted at 2026-08-10 14:37:55 on dist-test-slave-h6n0"
server_key: "fca903eaf338e9f8d51477ccdfe38636"
server_key_iv: "2c9ddd6313392afbeb9d7f05648c9e3b"
server_key_version: "encryptionkey@0"
I20260810 14:37:55.527092 26622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:55.545722 26622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:55.551409 26622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:55.551767 26622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:55.553244 26622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:55.555302 26622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:55.555454 26622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:55.555645 26622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:55.555727 26622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:55.600849 26622 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:37079
I20260810 14:37:55.600940 26824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:37079 every 8 connection(s)
I20260810 14:37:55.602735 26622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:37:55.604188 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 26622
I20260810 14:37:55.604437 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:37:55.605307 21725 external_mini_cluster.cc:1428] Setting key d68329c0d912c3d2ff3e5de6f5c9ac1c
Aug 10 14:37:55 dist-test-slave-h6n0 krb5kdc[26456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372675, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:55.629036 26828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:55.606267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:52773 (local address 127.21.55.126:43917)
0810 14:37:55.606812 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:55.606818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:55.607891 (+ 1073us) server_negotiation.cc:407] Connection header received
0810 14:37:55.610245 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:55.610252 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:55.610390 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:55.610529 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:55.612323 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:55.613036 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:55.614634 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:55.615039 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:55.615182 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:55.620610 (+ 5428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:55.620616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:55.620619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:55.620652 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:55.624267 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:55.625425 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:55.625430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:55.625432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:55.625661 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:55.626650 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:55.626654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:55.626656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:55.626953 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:55.627082 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:55.628317 (+ 1235us) server_negotiation.cc:299] Negotiation successful
0810 14:37:55.628654 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":914,"threads_started":1}
I20260810 14:37:55.630246 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:55.606831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43917 (local address 127.21.55.65:52773)
0810 14:37:55.607743 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:55.607782 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:55.609804 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:55.610938 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:55.610948 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:55.611537 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:55.612117 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:55.612135 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:55.613221 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:55.613225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:55.614455 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:55.614468 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:55.615274 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:55.615284 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:55.615532 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:55.616413 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:55.616448 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:55.620317 (+ 3869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:55.624488 (+ 4171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:55.624501 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:55.624521 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:55.625226 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:55.625825 (+ 599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:55.625832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:55.625839 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:55.626497 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:55.627104 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:55.627115 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:55.627644 (+ 529us) client_negotiation.cc:769] Sending connection context
0810 14:37:55.628011 (+ 367us) client_negotiation.cc:240] Negotiation successful
0810 14:37:55.628676 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":376,"threads_started":1}
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
I20260810 14:37:55.632539 26825 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43917
I20260810 14:37:55.632884 26825 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
I20260810 14:37:55.633880 26825 heartbeater.cc:511] Master 127.21.55.126:43917 requested a full tablet report, sending...
I20260810 14:37:55.636225 26526 ts_manager.cc:194] Registered new tserver with Master: 5e02a00bd88444a58924d721d1db857f (127.21.55.65:37079)
I20260810 14:37:55.637660 26526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:52773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:37:55.662640 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43917
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:56.046970 26833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:56.047521 26833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:56.047678 26833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:56.057322 26833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:56.057430 26833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:56.057541 26833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:37:56.069689 26833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:56.071070 26833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:56.072386 26833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:56.087437 26841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:56.087500 26842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:56.087687 26844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:56.088433 26833 server_base.cc:1034] running on GCE node
I20260810 14:37:56.089150 26833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:56.090171 26833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:56.091400 26833 hybrid_clock.cc:648] HybridClock initialized: now 1786372676091337 us; error 48 us; skew 500 ppm
I20260810 14:37:56.091761 26833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:56 dist-test-slave-h6n0 krb5kdc[26456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372676, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:56.095662 26833 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:37:56.097494 26833 webserver.cc:460] Webserver started at http://127.21.55.66:36381/ using document root <none> and password file <none>
I20260810 14:37:56.098265 26833 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:56.098428 26833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:56.098784 26833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:56.101686 26833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "99c1eb15a133407ebaa8098659b06eb9"
format_stamp: "Formatted at 2026-08-10 14:37:56 on dist-test-slave-h6n0"
server_key: "1900c288db1dbd7cff7a0b4b95ad7b1d"
server_key_iv: "f09114d0d35ec747af405f64d6130f47"
server_key_version: "encryptionkey@0"
I20260810 14:37:56.102519 26833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "99c1eb15a133407ebaa8098659b06eb9"
format_stamp: "Formatted at 2026-08-10 14:37:56 on dist-test-slave-h6n0"
server_key: "1900c288db1dbd7cff7a0b4b95ad7b1d"
server_key_iv: "f09114d0d35ec747af405f64d6130f47"
server_key_version: "encryptionkey@0"
I20260810 14:37:56.107921 26833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:37:56.111389 26857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:56.112833 26833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:37:56.113022 26833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "99c1eb15a133407ebaa8098659b06eb9"
format_stamp: "Formatted at 2026-08-10 14:37:56 on dist-test-slave-h6n0"
server_key: "1900c288db1dbd7cff7a0b4b95ad7b1d"
server_key_iv: "f09114d0d35ec747af405f64d6130f47"
server_key_version: "encryptionkey@0"
I20260810 14:37:56.113226 26833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:56.129590 26833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:56.135028 26833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:56.135296 26833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:56.136256 26833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:56.137713 26833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:56.137789 26833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:56.137884 26833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:56.137934 26833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:56.177435 26833 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:45421
I20260810 14:37:56.177474 27045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:45421 every 8 connection(s)
I20260810 14:37:56.178915 26833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:37:56.186115 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 26833
I20260810 14:37:56.186321 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:37:56.186878 21725 external_mini_cluster.cc:1428] Setting key 332ae8a2f1379756d5502161bf875137
Aug 10 14:37:56 dist-test-slave-h6n0 krb5kdc[26456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372676, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:56.202596 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:56.182405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:52827 (local address 127.21.55.126:43917)
0810 14:37:56.183072 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:56.183078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:56.183996 (+ 918us) server_negotiation.cc:407] Connection header received
0810 14:37:56.185844 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:56.185850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:56.185952 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:56.186087 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:56.187840 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.188736 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.190429 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.190966 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.191125 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:56.195319 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:56.195324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:56.195328 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:56.195358 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:56.199017 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.199787 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.199795 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.199801 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.199963 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.200482 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.200491 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.200497 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.200873 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:56.201061 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:56.201775 (+ 714us) server_negotiation.cc:299] Negotiation successful
0810 14:37:56.202045 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":359,"threads_started":1}
I20260810 14:37:56.203227 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:56.183049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43917 (local address 127.21.55.66:52827)
0810 14:37:56.183815 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:56.183864 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:56.185547 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:56.186321 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:56.186330 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:56.186940 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:56.187669 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:56.187694 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.188909 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.188914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:56.190280 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:56.190293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.191211 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.191220 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:56.191456 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:56.192168 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:56.192190 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:56.195094 (+ 2904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:56.199186 (+ 4092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:56.199193 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:56.199204 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:56.199642 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:56.200089 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:56.200093 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:56.200095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:56.200337 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:56.201039 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:56.201045 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:56.201369 (+ 324us) client_negotiation.cc:769] Sending connection context
0810 14:37:56.201610 (+ 241us) client_negotiation.cc:240] Negotiation successful
0810 14:37:56.202003 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":306,"threads_started":1}
I20260810 14:37:56.204814 27046 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43917
I20260810 14:37:56.205180 27046 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:56.206111 27046 heartbeater.cc:511] Master 127.21.55.126:43917 requested a full tablet report, sending...
I20260810 14:37:56.207852 26526 ts_manager.cc:194] Registered new tserver with Master: 99c1eb15a133407ebaa8098659b06eb9 (127.21.55.66:45421)
I20260810 14:37:56.208696 26526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:52827
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:37:56.238121 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43917
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:39385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:56.578848 27056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:56.579175 27056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:56.579356 27056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:56.586403 27056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:56.586577 27056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:56.586776 27056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:37:56.596460 27056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:56.597827 27056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:56.600417 27056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:56.616073 27065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:56.616101 27063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:56.616042 27072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:56.616953 27056 server_base.cc:1034] running on GCE node
I20260810 14:37:56.617616 27056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:56.618832 27056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:56.620213 27056 hybrid_clock.cc:648] HybridClock initialized: now 1786372676620111 us; error 66 us; skew 500 ppm
I20260810 14:37:56.620781 27056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:56 dist-test-slave-h6n0 krb5kdc[26456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372676, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:56.624449 27056 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:37:56.626192 27056 webserver.cc:460] Webserver started at http://127.21.55.67:45325/ using document root <none> and password file <none>
I20260810 14:37:56.627092 27056 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:56.627241 27056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:56.627565 27056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:56.630517 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "1b615fb176de431688713a06ec7ec852"
format_stamp: "Formatted at 2026-08-10 14:37:56 on dist-test-slave-h6n0"
server_key: "3f4240ad0a1f8daae705db2e437f10ae"
server_key_iv: "4b6b9c54d3b3a0e91dd77bdd29b8c5f2"
server_key_version: "encryptionkey@0"
I20260810 14:37:56.631421 27056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "1b615fb176de431688713a06ec7ec852"
format_stamp: "Formatted at 2026-08-10 14:37:56 on dist-test-slave-h6n0"
server_key: "3f4240ad0a1f8daae705db2e437f10ae"
server_key_iv: "4b6b9c54d3b3a0e91dd77bdd29b8c5f2"
server_key_version: "encryptionkey@0"
I20260810 14:37:56.637059 27056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 14:37:56.640669 27082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:56.641871 26825 heartbeater.cc:503] Master 127.21.55.126:43917 was elected leader, sending a full tablet report...
I20260810 14:37:56.642347 27056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:37:56.642633 27056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "1b615fb176de431688713a06ec7ec852"
format_stamp: "Formatted at 2026-08-10 14:37:56 on dist-test-slave-h6n0"
server_key: "3f4240ad0a1f8daae705db2e437f10ae"
server_key_iv: "4b6b9c54d3b3a0e91dd77bdd29b8c5f2"
server_key_version: "encryptionkey@0"
I20260810 14:37:56.642808 27056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:56.669965 27056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:56.674845 27056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:56.675171 27056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:56.676190 27056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:56.677553 27056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:56.677629 27056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:56.677803 27056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:56.677855 27056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:56.715342 27056 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:37203
I20260810 14:37:56.715515 27275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:37203 every 8 connection(s)
I20260810 14:37:56.716668 27056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:37:56.718353 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 27056
I20260810 14:37:56.718636 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:37:56.719313 21725 external_mini_cluster.cc:1428] Setting key 15686a872035a780cd2ff10469553a84
Aug 10 14:37:56 dist-test-slave-h6n0 krb5kdc[26456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372676, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:56.741833 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:56.720372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:38765 (local address 127.21.55.126:43917)
0810 14:37:56.720916 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:56.720922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:56.722407 (+ 1485us) server_negotiation.cc:407] Connection header received
0810 14:37:56.724840 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:56.724846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:56.724950 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:56.725097 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:56.726729 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.727537 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.729341 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.729766 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.729906 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:56.734813 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:56.734819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:56.734823 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:56.734862 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:56.738469 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.739422 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.739427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.739437 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.739569 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.740025 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.740032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.740040 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.740317 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:56.740486 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:56.741221 (+ 735us) server_negotiation.cc:299] Negotiation successful
0810 14:37:56.741444 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":283,"threads_started":1}
I20260810 14:37:56.742888 27282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:56.721044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43917 (local address 127.21.55.67:38765)
0810 14:37:56.722207 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:56.722275 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:56.724557 (+ 2282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:56.725349 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:56.725358 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:56.725907 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:56.726492 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:56.726509 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.727723 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.727729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:56.729166 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:56.729185 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.730161 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.730218 (+ 57us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:56.730675 (+ 457us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:56.731612 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:56.731640 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:56.734443 (+ 2803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:56.738717 (+ 4274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:56.738734 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:56.738748 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:56.739279 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:56.739688 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:56.739693 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:56.739703 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:56.739896 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:56.740456 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:56.740464 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:56.740828 (+ 364us) client_negotiation.cc:769] Sending connection context
0810 14:37:56.741111 (+ 283us) client_negotiation.cc:240] Negotiation successful
0810 14:37:56.741678 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":458,"threads_started":1}
I20260810 14:37:56.744288 27276 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43917
I20260810 14:37:56.744577 27276 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:56.745272 27276 heartbeater.cc:511] Master 127.21.55.126:43917 requested a full tablet report, sending...
I20260810 14:37:56.746804 26526 ts_manager.cc:194] Registered new tserver with Master: 1b615fb176de431688713a06ec7ec852 (127.21.55.67:37203)
I20260810 14:37:56.747807 26526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:38765
I20260810 14:37:56.750070 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:37:56 dist-test-slave-h6n0 krb5kdc[26456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372676, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 14:37:56 dist-test-slave-h6n0 krb5kdc[26456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372676, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:56.803690 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:56.789149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39002 (local address 127.21.55.126:43917)
0810 14:37:56.789325 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:56.789330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:56.789782 (+ 452us) server_negotiation.cc:407] Connection header received
0810 14:37:56.789928 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:56.789932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:56.790003 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:56.790101 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:56.791636 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.792442 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.793768 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.794155 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.794250 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:56.798241 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:56.798246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:56.798250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:56.798281 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:56.801227 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.801914 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.801918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.801920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.802031 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.802385 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.802388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.802390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.802816 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:56.803002 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:56.803298 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 14:37:56.803454 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 14:37:56.804219 26526 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39002
I20260810 14:37:56.806181 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 26622
Aug 10 14:37:56 dist-test-slave-h6n0 krb5kdc[26456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372676, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:37:56.822378 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:56.807376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.21.55.65:37079)
0810 14:37:56.807988 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:56.807995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:56.808019 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:37:56.808093 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:56.808099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:56.808296 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:56.808428 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:56.809497 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.810501 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.811940 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.812513 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.812662 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:56.816628 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:56.816640 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:56.816653 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:56.816684 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:56.819608 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.820269 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.820276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.820279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.820410 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.820788 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.820793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.820796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.821147 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:56.821470 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:56.821742 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 14:37:56.822017 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":302,"threads_started":1}
W20260810 14:37:56.823688 26819 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40854
W20260810 14:37:56.824764 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 26622: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:37:56.824947 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 26622
I20260810 14:37:56.854084 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 26833
Aug 10 14:37:56 dist-test-slave-h6n0 krb5kdc[26456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372676, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:37:56.874435 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:56.855141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44992 (local address 127.21.55.66:45421)
0810 14:37:56.855743 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:56.855750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:56.855777 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:37:56.855840 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:56.855846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:56.856060 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:56.856189 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:56.857715 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.859159 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.860989 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.861520 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.861636 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:56.867385 (+ 5749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:56.867408 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:56.867428 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:56.867481 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:56.871390 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.872152 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.872159 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.872162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.872304 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.872733 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.872736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.872738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.873090 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:56.873339 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:56.873727 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 14:37:56.874073 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":342,"threads_started":1}
W20260810 14:37:56.875801 27038 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44992
W20260810 14:37:56.876868 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 26833: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:37:56.877068 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 26833
I20260810 14:37:56.908141 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 27056
Aug 10 14:37:56 dist-test-slave-h6n0 krb5kdc[26456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372676, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:37:56.924721 27295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:56.909088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38438 (local address 127.21.55.67:37203)
0810 14:37:56.909712 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:56.909720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:56.909748 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:37:56.909810 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:56.909816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:56.910020 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:56.910205 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:56.911614 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.912669 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.913972 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:56.914658 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:56.914793 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:56.919174 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:56.919185 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:56.919199 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:56.919235 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:56.921903 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.922593 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.922607 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.922609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.922731 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:56.923105 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:56.923110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:56.923113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:56.923448 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:56.923771 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:56.924089 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 14:37:56.924368 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":306,"threads_started":1}
W20260810 14:37:56.926007 27269 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38438
W20260810 14:37:56.927256 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 27056: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:37:56.927412 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 27056
I20260810 14:37:56.952734 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 26481
I20260810 14:37:56.954255 26571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:37:57.060545 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 26481
2026-08-10T14:37:57Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4145 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:37:57 dist-test-slave-h6n0 krb5kdc[27299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:37:57 dist-test-slave-h6n0 krb5kdc[27299](info): set up 2 sockets
Aug 10 14:37:57 dist-test-slave-h6n0 krb5kdc[27299](info): commencing operation
krb5kdc: starting...
W20260810 14:37:58.452332 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:37:58 dist-test-slave-h6n0 krb5kdc[27299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:37:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:37:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:37:58.715600 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:40107
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:40107
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:59.072968 27324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:59.073482 27324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:59.073606 27324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:59.083122 27324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:37:59.083598 27324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:59.083772 27324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:59.083863 27324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:37:59.083909 27324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:37:59.096069 27324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:40107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:40107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:59.097575 27324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:59.098959 27324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:59.113195 27330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:59.113152 27336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:59.113241 27333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:59.116779 27324 server_base.cc:1034] running on GCE node
I20260810 14:37:59.118054 27324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:59.120200 27324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:59.121493 27324 hybrid_clock.cc:648] HybridClock initialized: now 1786372679121415 us; error 61 us; skew 500 ppm
I20260810 14:37:59.122008 27324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:59 dist-test-slave-h6n0 krb5kdc[27299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372679, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:59.126840 27324 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:37:59.128816 27324 webserver.cc:460] Webserver started at http://127.21.55.126:45173/ using document root <none> and password file <none>
I20260810 14:37:59.129597 27324 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:59.129760 27324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:59.130108 27324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:59.133272 27324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "ffd09e762f824ec3ae625a044b934729"
format_stamp: "Formatted at 2026-08-10 14:37:59 on dist-test-slave-h6n0"
server_key: "913b5155ed3ede8a485d68d33e062a35"
server_key_iv: "ebd582178f0beb3fc858eeb5630e45f6"
server_key_version: "encryptionkey@0"
I20260810 14:37:59.134096 27324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "ffd09e762f824ec3ae625a044b934729"
format_stamp: "Formatted at 2026-08-10 14:37:59 on dist-test-slave-h6n0"
server_key: "913b5155ed3ede8a485d68d33e062a35"
server_key_iv: "ebd582178f0beb3fc858eeb5630e45f6"
server_key_version: "encryptionkey@0"
I20260810 14:37:59.139895 27324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:37:59.143745 27349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:59.145222 27324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:37:59.145447 27324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "ffd09e762f824ec3ae625a044b934729"
format_stamp: "Formatted at 2026-08-10 14:37:59 on dist-test-slave-h6n0"
server_key: "913b5155ed3ede8a485d68d33e062a35"
server_key_iv: "ebd582178f0beb3fc858eeb5630e45f6"
server_key_version: "encryptionkey@0"
I20260810 14:37:59.145709 27324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:59.195066 27324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:59.200228 27324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:59.200567 27324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:59.222112 27324 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:40107
I20260810 14:37:59.222117 27450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:40107 every 8 connection(s)
I20260810 14:37:59.223549 27324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:37:59.224364 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 27324
I20260810 14:37:59.224658 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:37:59.225791 21725 external_mini_cluster.cc:1428] Setting key bb117b7fc714f4a0627742f9142c001f
I20260810 14:37:59.228626 27451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 14:37:59 dist-test-slave-h6n0 krb5kdc[27299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:59.241480 27451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729: Bootstrap starting.
I20260810 14:37:59.245275 27451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729: Neither blocks nor log segments found. Creating new log.
I20260810 14:37:59.246595 27451 log.cc:824] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729: Log is configured to *not* fsync() on all Append() calls
I20260810 14:37:59.250419 27451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729: No bootstrap required, opened a new log
I20260810 14:37:59.252372 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:59.228966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42064 (local address 127.21.55.126:40107)
0810 14:37:59.229755 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:59.229766 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:59.229805 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:37:59.231492 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:59.231512 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:59.231888 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:59.232284 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:59.233742 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:59.235166 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:59.236407 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:59.237062 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:59.237278 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:59.241910 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:59.241922 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:59.241946 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:59.241991 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:59.246652 (+ 4661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:59.248005 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:59.248045 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:59.248075 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:59.248380 (+ 305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:59.249105 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:59.249110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:59.249112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:59.250122 (+ 1010us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:59.250423 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:59.250926 (+ 503us) server_negotiation.cc:299] Negotiation successful
0810 14:37:59.251330 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":361,"threads_started":1}
I20260810 14:37:59.259857 27451 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd09e762f824ec3ae625a044b934729" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40107 } }
I20260810 14:37:59.260196 27451 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:37:59.260362 27451 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd09e762f824ec3ae625a044b934729, State: Initialized, Role: FOLLOWER
I20260810 14:37:59.260937 27451 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd09e762f824ec3ae625a044b934729" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40107 } }
I20260810 14:37:59.261173 27451 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:37:59.261309 27451 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:37:59.261497 27451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:37:59.263913 27451 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd09e762f824ec3ae625a044b934729" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40107 } }
I20260810 14:37:59.264438 27451 leader_election.cc:302] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffd09e762f824ec3ae625a044b934729; no voters:
I20260810 14:37:59.265022 27451 leader_election.cc:288] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:37:59.265216 27460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:37:59.266065 27460 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [term 1 LEADER]: Becoming Leader. State: Replica: ffd09e762f824ec3ae625a044b934729, State: Running, Role: LEADER
I20260810 14:37:59.266762 27460 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd09e762f824ec3ae625a044b934729" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40107 } }
I20260810 14:37:59.267488 27451 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:37:59.270275 27461 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffd09e762f824ec3ae625a044b934729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd09e762f824ec3ae625a044b934729" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40107 } } }
I20260810 14:37:59.270778 27461 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:59.270263 27462 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffd09e762f824ec3ae625a044b934729. Latest consensus state: current_term: 1 leader_uuid: "ffd09e762f824ec3ae625a044b934729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd09e762f824ec3ae625a044b934729" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40107 } } }
I20260810 14:37:59.271085 27462 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729 [sys.catalog]: This master's current role is: LEADER
I20260810 14:37:59.280280 27470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:37:59.285727 27470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:37:59.295218 27470 catalog_manager.cc:1347] Generated new cluster ID: af450a1ba56d4279909afd0339cb9dd8
I20260810 14:37:59.295399 27470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:37:59.308344 27470 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:37:59.310117 27470 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:37:59.326365 27470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffd09e762f824ec3ae625a044b934729: Generated new TSK 0
I20260810 14:37:59.327448 27470 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:37:59.407672 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40107
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:37:59.775025 27495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:37:59.775671 27495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:37:59.775931 27495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:37:59.784773 27495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:37:59.784893 27495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:37:59.785089 27495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:37:59.796042 27495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:37:59.797387 27495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:37:59.798758 27495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:37:59.815974 27510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:59.816049 27505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:37:59.815984 27506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:37:59.817247 27495 server_base.cc:1034] running on GCE node
I20260810 14:37:59.817950 27495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:37:59.819204 27495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:37:59.820544 27495 hybrid_clock.cc:648] HybridClock initialized: now 1786372679820463 us; error 60 us; skew 500 ppm
I20260810 14:37:59.821202 27495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:37:59 dist-test-slave-h6n0 krb5kdc[27299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372679, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:37:59.828848 27495 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:37:59.830910 27495 webserver.cc:460] Webserver started at http://127.21.55.65:42649/ using document root <none> and password file <none>
I20260810 14:37:59.831684 27495 fs_manager.cc:359] Metadata directory not provided
I20260810 14:37:59.831796 27495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:37:59.832082 27495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:37:59.835402 27495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "f7eac54246b7498e8338a5a965e88e15"
format_stamp: "Formatted at 2026-08-10 14:37:59 on dist-test-slave-h6n0"
server_key: "ed56758368b7503540bb547770890925"
server_key_iv: "17b9148cd60c582c8ef43a578a2dd86f"
server_key_version: "encryptionkey@0"
I20260810 14:37:59.836244 27495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "f7eac54246b7498e8338a5a965e88e15"
format_stamp: "Formatted at 2026-08-10 14:37:59 on dist-test-slave-h6n0"
server_key: "ed56758368b7503540bb547770890925"
server_key_iv: "17b9148cd60c582c8ef43a578a2dd86f"
server_key_version: "encryptionkey@0"
I20260810 14:37:59.841842 27495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:37:59.846387 27518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:59.848109 27495 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 14:37:59.848268 27495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "f7eac54246b7498e8338a5a965e88e15"
format_stamp: "Formatted at 2026-08-10 14:37:59 on dist-test-slave-h6n0"
server_key: "ed56758368b7503540bb547770890925"
server_key_iv: "17b9148cd60c582c8ef43a578a2dd86f"
server_key_version: "encryptionkey@0"
I20260810 14:37:59.848471 27495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:37:59.912674 27495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:37:59.918282 27495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:37:59.918643 27495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:37:59.919749 27495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:37:59.921252 27495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:37:59.921329 27495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:59.921419 27495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:37:59.921460 27495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:37:59.966282 27495 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:39665
I20260810 14:37:59.966346 27722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:39665 every 8 connection(s)
I20260810 14:37:59.967864 27495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:37:59.975111 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 27495
I20260810 14:37:59.975363 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:37:59.976394 21725 external_mini_cluster.cc:1428] Setting key c77c5fa9429d7a1f6a917e5d5aa3230f
Aug 10 14:37:59 dist-test-slave-h6n0 krb5kdc[27299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372679, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:37:59.992662 27728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:59.971084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:47339 (local address 127.21.55.126:40107)
0810 14:37:59.971786 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0810 14:37:59.971793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:37:59.972852 (+ 1059us) server_negotiation.cc:407] Connection header received
0810 14:37:59.974932 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:37:59.974938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:37:59.975045 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:37:59.975168 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:37:59.976929 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:59.977878 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:59.980814 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:37:59.981227 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:37:59.981363 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:59.985999 (+ 4636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:37:59.986005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:37:59.986008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:37:59.986045 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:37:59.988924 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:59.989753 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:59.989767 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:59.989769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:59.989900 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:37:59.990404 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:37:59.990418 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:37:59.990421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:37:59.990773 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:37:59.990947 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 14:37:59.991982 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0810 14:37:59.992246 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":295,"threads_started":1}
I20260810 14:37:59.993758 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:37:59.971744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:40107 (local address 127.21.55.65:47339)
0810 14:37:59.972663 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0810 14:37:59.972701 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 14:37:59.974598 (+ 1897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:37:59.975414 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:37:59.975424 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:37:59.976105 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:37:59.976717 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:59.976743 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:59.978072 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:59.978076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:59.980579 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:37:59.980599 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:37:59.981489 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:37:59.981499 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:37:59.981850 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:37:59.982757 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:37:59.982793 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:37:59.985730 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:37:59.989092 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:59.989103 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:59.989121 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:59.989604 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:59.990025 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:37:59.990031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:37:59.990035 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:37:59.990260 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:37:59.990919 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:37:59.990928 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:37:59.991303 (+ 375us) client_negotiation.cc:769] Sending connection context
0810 14:37:59.991842 (+ 539us) client_negotiation.cc:240] Negotiation successful
0810 14:37:59.992482 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":362,"threads_started":1}
I20260810 14:37:59.995587 27724 heartbeater.cc:347] Connected to a master server at 127.21.55.126:40107
I20260810 14:37:59.995975 27724 heartbeater.cc:464] Registering TS with master...
I20260810 14:37:59.997099 27724 heartbeater.cc:511] Master 127.21.55.126:40107 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
I20260810 14:37:59.999805 27376 ts_manager.cc:194] Registered new tserver with Master: f7eac54246b7498e8338a5a965e88e15 (127.21.55.65:39665)
I20260810 14:38:00.001330 27376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:47339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:38:00.038889 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40107
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:00.404024 27733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:00.404321 27733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:00.404482 27733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:00.412058 27733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:00.412180 27733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:00.412355 27733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:38:00.422776 27733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:00.424095 27733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:00.425550 27733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:00.439648 27739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:00.439549 27740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:00.439589 27744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:00.440562 27733 server_base.cc:1034] running on GCE node
I20260810 14:38:00.441264 27733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:00.442476 27733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:00.443804 27733 hybrid_clock.cc:648] HybridClock initialized: now 1786372680443757 us; error 51 us; skew 500 ppm
I20260810 14:38:00.444605 27733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:00 dist-test-slave-h6n0 krb5kdc[27299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:00.448740 27733 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:38:00.450755 27733 webserver.cc:460] Webserver started at http://127.21.55.66:33247/ using document root <none> and password file <none>
I20260810 14:38:00.451494 27733 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:00.451654 27733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:00.451987 27733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:00.455320 27733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "64b22465ce994ec5978315890f124fd1"
format_stamp: "Formatted at 2026-08-10 14:38:00 on dist-test-slave-h6n0"
server_key: "db5bc545cf6a72c44bbbb78db124fc36"
server_key_iv: "82b7fe4733719a17dbbf02f7f6cd7ce6"
server_key_version: "encryptionkey@0"
I20260810 14:38:00.456144 27733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "64b22465ce994ec5978315890f124fd1"
format_stamp: "Formatted at 2026-08-10 14:38:00 on dist-test-slave-h6n0"
server_key: "db5bc545cf6a72c44bbbb78db124fc36"
server_key_iv: "82b7fe4733719a17dbbf02f7f6cd7ce6"
server_key_version: "encryptionkey@0"
I20260810 14:38:00.461866 27733 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 14:38:00.466432 27756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:00.468423 27733 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:38:00.468616 27733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "64b22465ce994ec5978315890f124fd1"
format_stamp: "Formatted at 2026-08-10 14:38:00 on dist-test-slave-h6n0"
server_key: "db5bc545cf6a72c44bbbb78db124fc36"
server_key_iv: "82b7fe4733719a17dbbf02f7f6cd7ce6"
server_key_version: "encryptionkey@0"
I20260810 14:38:00.468844 27733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:00.520187 27733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:00.526805 27733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:00.527232 27733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:00.528542 27733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:00.530458 27733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:00.530652 27733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260810 14:38:00.530827 27733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:00.530895 27733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:00.569464 27733 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:39473
I20260810 14:38:00.569659 27950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:39473 every 8 connection(s)
I20260810 14:38:00.571098 27733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:38:00.575649 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 27733
I20260810 14:38:00.575866 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:38:00.576362 21725 external_mini_cluster.cc:1428] Setting key f171ef6fe54058ee61919da79b0ed61c
Aug 10 14:38:00 dist-test-slave-h6n0 krb5kdc[27299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372680, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:00.596655 27953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:00.573871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:41003 (local address 127.21.55.126:40107)
0810 14:38:00.574358 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:00.574362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:00.575146 (+ 784us) server_negotiation.cc:407] Connection header received
0810 14:38:00.577237 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:00.577241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:00.577319 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:00.577499 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:00.579299 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:00.580064 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:00.581644 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:00.582122 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:00.582263 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:00.586233 (+ 3970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:00.586247 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:00.586255 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:00.586327 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:00.591736 (+ 5409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:00.592795 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:00.592807 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:00.592814 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:00.593232 (+ 418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:00.594106 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:00.594115 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:00.594121 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:00.594595 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:00.594766 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:00.595781 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0810 14:38:00.596152 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":273,"threads_started":1}
I20260810 14:38:00.597338 27954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:00.574262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:40107 (local address 127.21.55.66:41003)
0810 14:38:00.574985 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:00.575036 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:00.576944 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:00.577748 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:00.577756 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:00.578454 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:00.579117 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:00.579135 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:00.580224 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:00.580231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:00.581483 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:00.581504 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:00.582362 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:00.582369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:00.582673 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:00.583413 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:00.583436 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:00.585836 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:00.591976 (+ 6140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:00.591984 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:00.591998 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:00.592490 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:00.593490 (+ 1000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:00.593497 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:00.593500 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:00.593910 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:00.594973 (+ 1063us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:00.594982 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:00.595342 (+ 360us) client_negotiation.cc:769] Sending connection context
0810 14:38:00.595615 (+ 273us) client_negotiation.cc:240] Negotiation successful
0810 14:38:00.596061 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":317,"threads_started":1}
I20260810 14:38:00.599048 27951 heartbeater.cc:347] Connected to a master server at 127.21.55.126:40107
I20260810 14:38:00.599395 27951 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:00.600185 27951 heartbeater.cc:511] Master 127.21.55.126:40107 requested a full tablet report, sending...
I20260810 14:38:00.601756 27376 ts_manager.cc:194] Registered new tserver with Master: 64b22465ce994ec5978315890f124fd1 (127.21.55.66:39473)
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
I20260810 14:38:00.602696 27376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:41003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:38:00.632581 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40107
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:01.000177 27960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:01.000479 27960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:01.000576 27960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 14:38:01.005065 27724 heartbeater.cc:503] Master 127.21.55.126:40107 was elected leader, sending a full tablet report...
W20260810 14:38:01.008419 27960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:01.008523 27960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:01.008731 27960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:38:01.018502 27960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:01.019814 27960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:01.021157 27960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:01.036459 27966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:01.036396 27968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:01.036473 27970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:01.037529 27960 server_base.cc:1034] running on GCE node
I20260810 14:38:01.038756 27960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:01.040102 27960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:01.041517 27960 hybrid_clock.cc:648] HybridClock initialized: now 1786372681041426 us; error 66 us; skew 500 ppm
I20260810 14:38:01.041973 27960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:01 dist-test-slave-h6n0 krb5kdc[27299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372681, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:01.046413 27960 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:38:01.048833 27960 webserver.cc:460] Webserver started at http://127.21.55.67:34691/ using document root <none> and password file <none>
I20260810 14:38:01.050110 27960 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:01.050307 27960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:01.050726 27960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:01.055204 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "9bfe3dd0810b4ad5aa75657a4f5aae2f"
format_stamp: "Formatted at 2026-08-10 14:38:01 on dist-test-slave-h6n0"
server_key: "8fee544501edac838cddcd244bfde528"
server_key_iv: "478cf616abf2f62267cfe02b541a4c5d"
server_key_version: "encryptionkey@0"
I20260810 14:38:01.056141 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "9bfe3dd0810b4ad5aa75657a4f5aae2f"
format_stamp: "Formatted at 2026-08-10 14:38:01 on dist-test-slave-h6n0"
server_key: "8fee544501edac838cddcd244bfde528"
server_key_iv: "478cf616abf2f62267cfe02b541a4c5d"
server_key_version: "encryptionkey@0"
I20260810 14:38:01.061754 27960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:38:01.066161 27983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:01.068681 27960 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260810 14:38:01.068933 27960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "9bfe3dd0810b4ad5aa75657a4f5aae2f"
format_stamp: "Formatted at 2026-08-10 14:38:01 on dist-test-slave-h6n0"
server_key: "8fee544501edac838cddcd244bfde528"
server_key_iv: "478cf616abf2f62267cfe02b541a4c5d"
server_key_version: "encryptionkey@0"
I20260810 14:38:01.069221 27960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:01.104833 27960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:01.111122 27960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:01.111508 27960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:01.112905 27960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:01.114375 27960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:01.114454 27960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:01.114607 27960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:01.114670 27960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260810 14:38:01.157908 27960 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:42565
I20260810 14:38:01.157997 28182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:42565 every 8 connection(s)
I20260810 14:38:01.160163 27960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:38:01.168807 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 27960
I20260810 14:38:01.169059 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:38:01.169670 21725 external_mini_cluster.cc:1428] Setting key a5c47e6f2bc786a9a6f7e70e61d7cf02
Aug 10 14:38:01 dist-test-slave-h6n0 krb5kdc[27299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372681, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:01.184227 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:01.163794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:45737 (local address 127.21.55.126:40107)
0810 14:38:01.164301 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:01.164306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:01.165158 (+ 852us) server_negotiation.cc:407] Connection header received
0810 14:38:01.167293 (+ 2135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:01.167299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:01.167403 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:01.167530 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:01.169814 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.170672 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.173457 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.173907 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.174013 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:01.177959 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:01.177964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:01.177968 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:01.178010 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:01.180995 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.181667 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.181670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.181673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.181783 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.182191 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.182196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.182198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.182607 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:01.182753 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:01.183453 (+ 700us) server_negotiation.cc:299] Negotiation successful
0810 14:38:01.183813 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":270,"threads_started":1}
I20260810 14:38:01.185210 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:01.164237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:40107 (local address 127.21.55.67:45737)
0810 14:38:01.164966 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:01.165042 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:01.166862 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:01.167802 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:01.167810 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:01.168647 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:01.169555 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:01.169582 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.170857 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.170861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:01.173271 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:01.173285 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.174154 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.174159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:01.174401 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:01.175232 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:01.175254 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:01.177720 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:01.181126 (+ 3406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:01.181133 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:01.181145 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:01.181566 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:01.181876 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:01.181879 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:01.181882 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:01.182059 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:01.182732 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:01.182739 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:01.183059 (+ 320us) client_negotiation.cc:769] Sending connection context
0810 14:38:01.183346 (+ 287us) client_negotiation.cc:240] Negotiation successful
0810 14:38:01.183798 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":299,"threads_started":1}
I20260810 14:38:01.186692 28183 heartbeater.cc:347] Connected to a master server at 127.21.55.126:40107
I20260810 14:38:01.186997 28183 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:01.187717 28183 heartbeater.cc:511] Master 127.21.55.126:40107 requested a full tablet report, sending...
I20260810 14:38:01.189218 27376 ts_manager.cc:194] Registered new tserver with Master: 9bfe3dd0810b4ad5aa75657a4f5aae2f (127.21.55.67:42565)
I20260810 14:38:01.190034 27376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:45737
I20260810 14:38:01.198671 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:38:01 dist-test-slave-h6n0 krb5kdc[27299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:38:01 dist-test-slave-h6n0 krb5kdc[27299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:38:01.259908 28196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:01.239460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60008 (local address 127.21.55.65:39665)
0810 14:38:01.240082 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:01.240090 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:01.240117 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:38:01.240181 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:01.240186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:01.240396 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:01.240542 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:01.242034 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.243276 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.244865 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.245745 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.245932 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:01.251242 (+ 5310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:01.251263 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:01.251286 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:01.251349 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:01.255808 (+ 4459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.256873 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.256882 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.256884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.257118 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.257639 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.257645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.257648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.258178 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:01.258603 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:01.259192 (+ 589us) server_negotiation.cc:299] Negotiation successful
0810 14:38:01.259453 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":313,"threads_started":1}
Aug 10 14:38:01 dist-test-slave-h6n0 krb5kdc[27299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:01.281899 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:01.266053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42074 (local address 127.21.55.126:40107)
0810 14:38:01.266314 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:01.266320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:01.266470 (+ 150us) server_negotiation.cc:407] Connection header received
0810 14:38:01.266768 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:01.266773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:01.266850 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:01.266960 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:01.268413 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.269189 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.270474 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.271011 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.271129 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:01.275972 (+ 4843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:01.275977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:01.275980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:01.276012 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:01.278986 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.279748 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.279752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.279754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.279874 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.280323 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.280327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.280329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.280675 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:01.280860 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:01.281365 (+ 505us) server_negotiation.cc:299] Negotiation successful
0810 14:38:01.281532 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
W20260810 14:38:01.282454 27376 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42074
I20260810 14:38:01.283699 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 27495
I20260810 14:38:01.299026 28196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:01.284838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60014 (local address 127.21.55.65:39665)
0810 14:38:01.285317 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:01.285325 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:01.285501 (+ 176us) server_negotiation.cc:407] Connection header received
0810 14:38:01.285708 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:01.285713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:01.285792 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:01.285908 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:01.287170 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.287830 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.289348 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.289773 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.289925 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:01.292224 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:01.292229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:01.292232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:01.292263 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:01.295630 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.296603 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.296610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.296614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.296797 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.297261 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.297265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.297267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.297563 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:01.298005 (+ 442us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:01.298358 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 14:38:01.298695 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260810 14:38:01.299687 27716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:01.389880 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 27495
I20260810 14:38:01.416072 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 27733
Aug 10 14:38:01 dist-test-slave-h6n0 krb5kdc[27299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:38:01.433720 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:01.417025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35660 (local address 127.21.55.66:39473)
0810 14:38:01.417566 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:01.417573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:01.417598 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:38:01.417659 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:01.417665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:01.417864 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:01.417996 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:01.419548 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.420703 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.422423 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.423162 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.423270 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:01.427732 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:01.427745 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:01.427759 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:01.427808 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:01.430702 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.431367 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.431374 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.431376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.431503 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.432265 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.432268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.432270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.432627 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:01.432869 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:01.433157 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 14:38:01.433373 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":258,"threads_started":1}
I20260810 14:38:01.435307 27940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:01.526403 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 27733
I20260810 14:38:01.557384 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 27960
Aug 10 14:38:01 dist-test-slave-h6n0 krb5kdc[27299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:38:01.575420 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:01.558490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45918 (local address 127.21.55.67:42565)
0810 14:38:01.559222 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:01.559229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:01.559260 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:38:01.559327 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:01.559333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:01.559555 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:01.559688 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:01.561049 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.562102 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.563655 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:01.564246 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:01.564361 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:01.569341 (+ 4980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:01.569353 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:01.569368 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:01.569401 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:01.572481 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.573128 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.573135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.573138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.573261 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:01.573580 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:01.573583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:01.573585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:01.573953 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:01.574285 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:01.574866 (+ 581us) server_negotiation.cc:299] Negotiation successful
0810 14:38:01.575073 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":346,"threads_started":1}
I20260810 14:38:01.576845 28178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:01.667963 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 27960
I20260810 14:38:01.692885 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 27324
I20260810 14:38:01.694131 27443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:01.799331 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 27324
2026-08-10T14:38:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4736 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:38:01 dist-test-slave-h6n0 krb5kdc[28222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:38:01 dist-test-slave-h6n0 krb5kdc[28222](info): set up 2 sockets
Aug 10 14:38:01 dist-test-slave-h6n0 krb5kdc[28222](info): commencing operation
krb5kdc: starting...
W20260810 14:38:03.200203 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:38:03 dist-test-slave-h6n0 krb5kdc[28222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:38:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:38:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:38:03.446391 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:41231
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:41231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:03.788492 28243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:03.788779 28243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:03.788872 28243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:03.795921 28243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:03.796026 28243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:03.796072 28243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:03.796131 28243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:03.796180 28243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:38:03.805625 28243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:41231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:41231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:03.806802 28243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:03.808096 28243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:03.819531 28260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:03.819557 28258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:03.819564 28253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:03.821035 28243 server_base.cc:1034] running on GCE node
I20260810 14:38:03.821813 28243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:03.823248 28243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:03.824635 28243 hybrid_clock.cc:648] HybridClock initialized: now 1786372683824538 us; error 37 us; skew 500 ppm
I20260810 14:38:03.825480 28243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:03 dist-test-slave-h6n0 krb5kdc[28222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372683, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:03.829656 28243 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:38:03.831674 28243 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:03.831784 28243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:03.832198 28243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:03.835404 28243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "c49828e1499f450085b0d911ef31885e"
format_stamp: "Formatted at 2026-08-10 14:38:03 on dist-test-slave-h6n0"
server_key: "9d697ea4234f98c8b8609152ae92d83f"
server_key_iv: "295be3e91260756e29c7bc9e11b8e239"
server_key_version: "encryptionkey@0"
I20260810 14:38:03.836253 28243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "c49828e1499f450085b0d911ef31885e"
format_stamp: "Formatted at 2026-08-10 14:38:03 on dist-test-slave-h6n0"
server_key: "9d697ea4234f98c8b8609152ae92d83f"
server_key_iv: "295be3e91260756e29c7bc9e11b8e239"
server_key_version: "encryptionkey@0"
I20260810 14:38:03.841557 28243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 14:38:03.845475 28267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:03.847296 28243 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260810 14:38:03.847517 28243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "c49828e1499f450085b0d911ef31885e"
format_stamp: "Formatted at 2026-08-10 14:38:03 on dist-test-slave-h6n0"
server_key: "9d697ea4234f98c8b8609152ae92d83f"
server_key_iv: "295be3e91260756e29c7bc9e11b8e239"
server_key_version: "encryptionkey@0"
I20260810 14:38:03.847751 28243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:03.883240 28243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:03.889582 28243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:03.889891 28243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:03.912082 28243 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:41231
I20260810 14:38:03.912103 28345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:41231 every 8 connection(s)
I20260810 14:38:03.913028 28243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:38:03.916630 28346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:03.920022 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 28243
I20260810 14:38:03.920732 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:38:03.921561 21725 external_mini_cluster.cc:1428] Setting key b743548e0965b2e2924abb7884b8f215
I20260810 14:38:03.932317 28346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e: Bootstrap starting.
I20260810 14:38:03.936286 28346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e: Neither blocks nor log segments found. Creating new log.
Aug 10 14:38:03 dist-test-slave-h6n0 krb5kdc[28222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:03.937526 28346 log.cc:824] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:03.941187 28346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e: No bootstrap required, opened a new log
I20260810 14:38:03.947749 28355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:03.924946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58656 (local address 127.21.55.126:41231)
0810 14:38:03.926177 (+ 1231us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:03.926193 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:03.926264 (+ 71us) server_negotiation.cc:407] Connection header received
0810 14:38:03.928335 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:03.928357 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:03.928767 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:03.929206 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:03.930806 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:03.932071 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:03.933428 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:03.934101 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:03.934311 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:03.938835 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:03.938849 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:03.938875 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:03.938909 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:03.942447 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:03.943445 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:03.943460 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:03.943469 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:03.943795 (+ 326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:03.944355 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:03.944359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:03.944361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:03.945188 (+ 827us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:03.945447 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:03.945964 (+ 517us) server_negotiation.cc:299] Negotiation successful
0810 14:38:03.946466 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":618,"threads_started":1}
I20260810 14:38:03.950510 28346 raft_consensus.cc:348] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49828e1499f450085b0d911ef31885e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41231 } }
I20260810 14:38:03.950963 28346 raft_consensus.cc:374] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:03.951092 28346 raft_consensus.cc:729] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c49828e1499f450085b0d911ef31885e, State: Initialized, Role: FOLLOWER
I20260810 14:38:03.951571 28346 consensus_queue.cc:260] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49828e1499f450085b0d911ef31885e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41231 } }
I20260810 14:38:03.951768 28346 raft_consensus.cc:388] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:38:03.951933 28346 raft_consensus.cc:482] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:38:03.952164 28346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:03.954342 28346 raft_consensus.cc:504] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49828e1499f450085b0d911ef31885e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41231 } }
I20260810 14:38:03.954873 28346 leader_election.cc:302] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c49828e1499f450085b0d911ef31885e; no voters:
I20260810 14:38:03.955744 28346 leader_election.cc:288] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:38:03.956007 28358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:03.957479 28358 raft_consensus.cc:686] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [term 1 LEADER]: Becoming Leader. State: Replica: c49828e1499f450085b0d911ef31885e, State: Running, Role: LEADER
I20260810 14:38:03.957968 28358 consensus_queue.cc:237] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49828e1499f450085b0d911ef31885e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41231 } }
I20260810 14:38:03.958257 28346 sys_catalog.cc:564] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:03.961112 28360 sys_catalog.cc:455] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c49828e1499f450085b0d911ef31885e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49828e1499f450085b0d911ef31885e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41231 } } }
I20260810 14:38:03.961449 28360 sys_catalog.cc:458] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:03.961872 28361 sys_catalog.cc:455] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c49828e1499f450085b0d911ef31885e. Latest consensus state: current_term: 1 leader_uuid: "c49828e1499f450085b0d911ef31885e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c49828e1499f450085b0d911ef31885e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41231 } } }
I20260810 14:38:03.962168 28361 sys_catalog.cc:458] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:03.964985 28376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:03.970070 28376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:03.980440 28376 catalog_manager.cc:1347] Generated new cluster ID: 7f000260b3ef4928ad4abdc9dc92e148
I20260810 14:38:03.980568 28376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:03.991582 28376 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:38:03.993501 28376 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:04.006738 28376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e: Generated new TSK 0
I20260810 14:38:04.007776 28376 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:38:04.070739 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41231
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:04.426672 28399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:04.426970 28399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:04.427174 28399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:04.434746 28399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:04.435065 28399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:04.435308 28399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:38:04.447662 28399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:04.449134 28399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:04.450490 28399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:04.463907 28407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:04.463990 28408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:04.464010 28412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:04.465265 28399 server_base.cc:1034] running on GCE node
I20260810 14:38:04.465909 28399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:04.466969 28399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:04.468199 28399 hybrid_clock.cc:648] HybridClock initialized: now 1786372684468144 us; error 30 us; skew 500 ppm
I20260810 14:38:04.468540 28399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:04 dist-test-slave-h6n0 krb5kdc[28222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372684, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:04.473531 28399 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:38:04.476253 28399 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:04.476380 28399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:04.476753 28399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:04.481703 28399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "9eba0550f3b042299ca21d7f8c52f9c1"
format_stamp: "Formatted at 2026-08-10 14:38:04 on dist-test-slave-h6n0"
server_key: "2a041360c54560d791a4c6f26b8f7745"
server_key_iv: "07870b97e18948afd231a1a7ed4e449d"
server_key_version: "encryptionkey@0"
I20260810 14:38:04.483340 28399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "9eba0550f3b042299ca21d7f8c52f9c1"
format_stamp: "Formatted at 2026-08-10 14:38:04 on dist-test-slave-h6n0"
server_key: "2a041360c54560d791a4c6f26b8f7745"
server_key_iv: "07870b97e18948afd231a1a7ed4e449d"
server_key_version: "encryptionkey@0"
I20260810 14:38:04.495345 28399 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.006s sys 0.001s
I20260810 14:38:04.500746 28426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:04.502656 28399 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:38:04.502862 28399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "9eba0550f3b042299ca21d7f8c52f9c1"
format_stamp: "Formatted at 2026-08-10 14:38:04 on dist-test-slave-h6n0"
server_key: "2a041360c54560d791a4c6f26b8f7745"
server_key_iv: "07870b97e18948afd231a1a7ed4e449d"
server_key_version: "encryptionkey@0"
I20260810 14:38:04.503201 28399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:04.544822 28399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:04.550135 28399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:04.550460 28399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:04.551717 28399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:04.553289 28399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:04.553391 28399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:04.553465 28399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:04.553539 28399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:04.598007 28399 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:37153
I20260810 14:38:04.598112 28615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:37153 every 8 connection(s)
I20260810 14:38:04.598966 28399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:38:04.600817 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 28399
I20260810 14:38:04.601128 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:38:04.602293 21725 external_mini_cluster.cc:1428] Setting key 002e394aef6f4afdbb8eecd841a55d6f
Aug 10 14:38:04 dist-test-slave-h6n0 krb5kdc[28222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372684, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
I20260810 14:38:04.627954 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:04.602204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:51679 (local address 127.21.55.126:41231)
0810 14:38:04.604185 (+ 1981us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:04.604192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:04.604362 (+ 170us) server_negotiation.cc:407] Connection header received
0810 14:38:04.606887 (+ 2525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:04.606892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:04.606990 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:04.607112 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:04.608930 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:04.609751 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:04.611418 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:04.611814 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:04.611959 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:04.616491 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:04.616498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:04.616502 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:04.616559 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:04.623997 (+ 7438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:04.625129 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:04.625134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:04.625137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:04.625264 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:04.625785 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:04.625788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:04.625790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:04.626062 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:04.626171 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:04.627275 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0810 14:38:04.627542 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1834,"thread_start_us":1696,"threads_started":1}
I20260810 14:38:04.628903 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:04.602744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:41231 (local address 127.21.55.65:51679)
0810 14:38:04.604128 (+ 1384us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:04.604194 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:04.606603 (+ 2409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:04.607351 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:04.607359 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:04.608012 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:04.608734 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:04.608755 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:04.609931 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:04.609935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:04.611248 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:04.611262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:04.612062 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:04.612071 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:04.612299 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:04.613118 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:04.613143 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:04.616133 (+ 2990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:04.624184 (+ 8051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:04.624197 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:04.624220 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:04.624976 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:04.625383 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:04.625388 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:04.625392 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:04.625664 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:04.626200 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:04.626211 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:04.626760 (+ 549us) client_negotiation.cc:769] Sending connection context
0810 14:38:04.627127 (+ 367us) client_negotiation.cc:240] Negotiation successful
0810 14:38:04.627718 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"thread_start_us":508,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
I20260810 14:38:04.630756 28617 heartbeater.cc:347] Connected to a master server at 127.21.55.126:41231
I20260810 14:38:04.631086 28617 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:04.631783 28617 heartbeater.cc:511] Master 127.21.55.126:41231 requested a full tablet report, sending...
I20260810 14:38:04.634028 28288 ts_manager.cc:194] Registered new tserver with Master: 9eba0550f3b042299ca21d7f8c52f9c1 (127.21.55.65:37153)
I20260810 14:38:04.636163 28288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:51679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:38:04.657683 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41231
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:05.028743 28626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:05.029035 28626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:05.029134 28626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:05.036409 28626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:05.036531 28626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:05.036650 28626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:38:05.046336 28626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:05.047705 28626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:05.049011 28626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:05.063134 28636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:05.063352 28634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:05.063400 28640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:05.064561 28626 server_base.cc:1034] running on GCE node
I20260810 14:38:05.065277 28626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:05.066249 28626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:05.067593 28626 hybrid_clock.cc:648] HybridClock initialized: now 1786372685067501 us; error 37 us; skew 500 ppm
I20260810 14:38:05.068095 28626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:05 dist-test-slave-h6n0 krb5kdc[28222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:05.075453 28626 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:38:05.076823 28626 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:05.076939 28626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:05.077513 28626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:05.082082 28626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "cb6de593a0ac4723a9da7b87f2136847"
format_stamp: "Formatted at 2026-08-10 14:38:05 on dist-test-slave-h6n0"
server_key: "496373066b4b725f3bac899eed7e7586"
server_key_iv: "de8b6481d658ab0501034d50f7933ba7"
server_key_version: "encryptionkey@0"
I20260810 14:38:05.083053 28626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "cb6de593a0ac4723a9da7b87f2136847"
format_stamp: "Formatted at 2026-08-10 14:38:05 on dist-test-slave-h6n0"
server_key: "496373066b4b725f3bac899eed7e7586"
server_key_iv: "de8b6481d658ab0501034d50f7933ba7"
server_key_version: "encryptionkey@0"
I20260810 14:38:05.088725 28626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:38:05.093712 28649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:05.095178 28626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 14:38:05.095348 28626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "cb6de593a0ac4723a9da7b87f2136847"
format_stamp: "Formatted at 2026-08-10 14:38:05 on dist-test-slave-h6n0"
server_key: "496373066b4b725f3bac899eed7e7586"
server_key_iv: "de8b6481d658ab0501034d50f7933ba7"
server_key_version: "encryptionkey@0"
I20260810 14:38:05.095577 28626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:05.128238 28626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:05.134166 28626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:05.134421 28626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:05.135507 28626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:05.136931 28626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:05.137025 28626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:05.137184 28626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:05.137274 28626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:05.172751 28626 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:32791
I20260810 14:38:05.172863 28841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:32791 every 8 connection(s)
I20260810 14:38:05.174116 28626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:38:05.181480 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 28626
I20260810 14:38:05.181756 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:38:05.182353 21725 external_mini_cluster.cc:1428] Setting key 6349592c416158751186a3b4c7545fac
Aug 10 14:38:05 dist-test-slave-h6n0 krb5kdc[28222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:05.201045 28844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:05.176797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:47059 (local address 127.21.55.126:41231)
0810 14:38:05.177286 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:05.177293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:05.178386 (+ 1093us) server_negotiation.cc:407] Connection header received
0810 14:38:05.180409 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:05.180415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:05.180498 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:05.180620 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:05.183296 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.184371 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.187884 (+ 3513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.188474 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.188611 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:05.194094 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:05.194100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:05.194104 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:05.194143 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:05.197596 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:05.198424 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:05.198430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:05.198433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:05.198643 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:05.199099 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:05.199104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:05.199107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:05.199421 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:05.199581 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:05.200420 (+ 839us) server_negotiation.cc:299] Negotiation successful
0810 14:38:05.200649 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":235,"threads_started":1}
I20260810 14:38:05.202142 28846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:05.177299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:41231 (local address 127.21.55.66:47059)
0810 14:38:05.178225 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:05.178272 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:05.180056 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:05.181365 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:05.181378 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:05.182199 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:05.183053 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:05.183073 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.184656 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.184665 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:05.187567 (+ 2902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:05.187604 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.188813 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.188824 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:05.189225 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:05.190278 (+ 1053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:05.190331 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:05.193784 (+ 3453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:05.197758 (+ 3974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:05.197771 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:05.197791 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:05.198279 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:05.198760 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:05.198765 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:05.198769 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:05.198970 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:05.199576 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:05.199584 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:05.200020 (+ 436us) client_negotiation.cc:769] Sending connection context
0810 14:38:05.200303 (+ 283us) client_negotiation.cc:240] Negotiation successful
0810 14:38:05.200871 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":430,"threads_started":1}
I20260810 14:38:05.203697 28842 heartbeater.cc:347] Connected to a master server at 127.21.55.126:41231
I20260810 14:38:05.204034 28842 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:05.204735 28842 heartbeater.cc:511] Master 127.21.55.126:41231 requested a full tablet report, sending...
I20260810 14:38:05.206275 28288 ts_manager.cc:194] Registered new tserver with Master: cb6de593a0ac4723a9da7b87f2136847 (127.21.55.66:32791)
I20260810 14:38:05.207158 28288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:47059
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:38:05.244220 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41231
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:05.617126 28850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:05.617422 28850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:05.617599 28850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:05.625267 28850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:05.625383 28850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:05.625576 28850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:38:05.635499 28850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:05.636837 28850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:05.638150 28850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:38:05.639386 28617 heartbeater.cc:503] Master 127.21.55.126:41231 was elected leader, sending a full tablet report...
W20260810 14:38:05.652117 28863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:05.652163 28865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:05.652069 28862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:05.653002 28850 server_base.cc:1034] running on GCE node
I20260810 14:38:05.653558 28850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:05.654441 28850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:05.655658 28850 hybrid_clock.cc:648] HybridClock initialized: now 1786372685655606 us; error 43 us; skew 500 ppm
I20260810 14:38:05.655936 28850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:05 dist-test-slave-h6n0 krb5kdc[28222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:05.659410 28850 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:38:05.660640 28850 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:05.660733 28850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:05.661006 28850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:05.664016 28850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "9f46caf9f72e41a9aa57a466222357a8"
format_stamp: "Formatted at 2026-08-10 14:38:05 on dist-test-slave-h6n0"
server_key: "bf76c32f4219880c4bd72316f4cf25c4"
server_key_iv: "6bb5bdbaf54ba456ef270d8c7d0ec731"
server_key_version: "encryptionkey@0"
I20260810 14:38:05.664814 28850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "9f46caf9f72e41a9aa57a466222357a8"
format_stamp: "Formatted at 2026-08-10 14:38:05 on dist-test-slave-h6n0"
server_key: "bf76c32f4219880c4bd72316f4cf25c4"
server_key_iv: "6bb5bdbaf54ba456ef270d8c7d0ec731"
server_key_version: "encryptionkey@0"
I20260810 14:38:05.670249 28850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 14:38:05.674111 28875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:05.676173 28850 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:38:05.676330 28850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "9f46caf9f72e41a9aa57a466222357a8"
format_stamp: "Formatted at 2026-08-10 14:38:05 on dist-test-slave-h6n0"
server_key: "bf76c32f4219880c4bd72316f4cf25c4"
server_key_iv: "6bb5bdbaf54ba456ef270d8c7d0ec731"
server_key_version: "encryptionkey@0"
I20260810 14:38:05.676520 28850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:05.691274 28850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:05.697748 28850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:05.698063 28850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:05.699553 28850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:05.701521 28850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:05.701633 28850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:05.701797 28850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:05.701884 28850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:05.744704 28850 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:36005
I20260810 14:38:05.744796 29055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:36005 every 8 connection(s)
I20260810 14:38:05.745682 28850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:38:05.752570 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 28850
I20260810 14:38:05.752835 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:38:05.753280 21725 external_mini_cluster.cc:1428] Setting key 955ce9056833a22661fd093cdee50fee
Aug 10 14:38:05 dist-test-slave-h6n0 krb5kdc[28222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372685, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:05.769817 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:05.748648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:55553 (local address 127.21.55.126:41231)
0810 14:38:05.749218 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:05.749222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:05.750176 (+ 954us) server_negotiation.cc:407] Connection header received
0810 14:38:05.752216 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:05.752224 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:05.752350 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:05.752500 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:05.754333 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.755627 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.757457 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.758306 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.758421 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:05.762376 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:05.762382 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:05.762386 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:05.762419 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:05.765599 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:05.766838 (+ 1239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:05.766854 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:05.766857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:05.766999 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:05.767518 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:05.767522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:05.767524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:05.767914 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:05.768209 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:05.769268 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0810 14:38:05.769529 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":313,"threads_started":1}
I20260810 14:38:05.770762 29066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:05.749231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:41231 (local address 127.21.55.67:55553)
0810 14:38:05.749957 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:05.750036 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:05.751913 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:05.752729 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:05.752738 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:05.753304 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:05.754015 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:05.754048 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.755786 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.755791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:05.757258 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:05.757271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.758525 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.758572 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:05.758779 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:05.759539 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:05.759567 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:05.762153 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:05.765819 (+ 3666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:05.765827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:05.765838 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:05.766277 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:05.767143 (+ 866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:05.767150 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:05.767153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:05.767369 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:05.768290 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:05.768298 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:05.768719 (+ 421us) client_negotiation.cc:769] Sending connection context
0810 14:38:05.769028 (+ 309us) client_negotiation.cc:240] Negotiation successful
0810 14:38:05.769505 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":306,"threads_started":1}
I20260810 14:38:05.772219 29056 heartbeater.cc:347] Connected to a master server at 127.21.55.126:41231
I20260810 14:38:05.772549 29056 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:05.773206 29056 heartbeater.cc:511] Master 127.21.55.126:41231 requested a full tablet report, sending...
I20260810 14:38:05.774928 28288 ts_manager.cc:194] Registered new tserver with Master: 9f46caf9f72e41a9aa57a466222357a8 (127.21.55.67:36005)
I20260810 14:38:05.775730 28288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:55553
I20260810 14:38:05.782599 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:38:05.801789 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:05.787416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.21.55.126:41231)
0810 14:38:05.787588 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:05.787593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:05.788161 (+ 568us) server_negotiation.cc:407] Connection header received
0810 14:38:05.788373 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:05.788377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:05.788438 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:05.788545 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:05.789970 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.790763 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.792511 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.792938 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.793074 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:05.795436 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:05.795441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:05.795444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:05.795473 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:05.798295 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:05.799093 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:05.799101 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:05.799105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:05.799240 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:05.800163 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:05.800168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:05.800172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:05.800539 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:05.800832 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:05.801199 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 14:38:05.801439 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:38:05.806945 28288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:38:05.813941 28288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:38:05.852727 29085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:05.827041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:36005 (local address 127.0.0.1:46238)
0810 14:38:05.838387 (+ 11346us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:05.838446 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:05.839441 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:05.843183 (+ 3742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:05.843194 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:05.843235 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:05.843716 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:05.843733 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.846284 (+ 2551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.846292 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:05.849277 (+ 2985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:05.849298 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.851659 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.851668 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:05.851936 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:05.852005 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 14:38:05.852116 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 14:38:05.852310 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11042,"threads_started":1}
I20260810 14:38:05.854492 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:05.827420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46238 (local address 127.21.55.67:36005)
0810 14:38:05.842440 (+ 15020us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:05.842453 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:05.842493 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:38:05.842629 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:05.842641 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:05.842912 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:05.843962 (+ 1050us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:05.844293 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.846108 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.849539 (+ 3431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.851409 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.853611 (+ 2202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:05.853853 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:05.853943 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 14:38:05.854108 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14805,"thread_start_us":14622,"threads_started":1}
I20260810 14:38:05.861455 28949 tablet_service.cc:1467] Processing CreateTablet for tablet ed89de3ebf5848068a7e2cd893d74697 (DEFAULT_TABLE table=test-table [id=8cc0f9db8d7e48a598fc23b3e2fe7bf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:05.862473 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:05.827523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:37153 (local address 127.0.0.1:59022)
0810 14:38:05.852997 (+ 25474us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:05.853016 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:05.853120 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:05.854572 (+ 1452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:05.854577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:05.854612 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:05.855045 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:05.855060 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.856952 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.856961 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:05.859668 (+ 2707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:05.859689 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.861551 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.861559 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:05.861767 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:05.861809 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 14:38:05.861918 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 14:38:05.862056 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25352,"thread_start_us":10414,"threads_started":1}
I20260810 14:38:05.862766 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:05.828192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59022 (local address 127.21.55.65:37153)
0810 14:38:05.853888 (+ 25696us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:05.853896 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:05.853924 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:38:05.854017 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:05.854025 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:05.854268 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:05.855055 (+ 787us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:05.855284 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.856801 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.859859 (+ 3058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.861302 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.861480 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:05.861728 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:05.862246 (+ 518us) server_negotiation.cc:299] Negotiation successful
0810 14:38:05.862419 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25515,"thread_start_us":25327,"threads_started":1}
I20260810 14:38:05.863344 29085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:05.827893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:32791 (local address 127.0.0.1:39298)
0810 14:38:05.853317 (+ 25424us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:05.853334 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:05.853437 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:05.855065 (+ 1628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:05.855069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:05.855098 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:05.855313 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:05.855325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.857600 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.857610 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:05.859859 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:05.859880 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.861971 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.861978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:05.862192 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:05.862231 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 14:38:05.862329 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 14:38:05.862507 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25313,"thread_start_us":25288,"threads_started":1}
I20260810 14:38:05.863883 29090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:05.828624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39298 (local address 127.21.55.66:32791)
0810 14:38:05.852296 (+ 23672us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:05.852305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:05.854213 (+ 1908us) server_negotiation.cc:407] Connection header received
0810 14:38:05.854333 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:05.854341 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:05.854589 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:05.854783 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:05.855514 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.857265 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.860239 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:05.861774 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:05.861930 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:05.862103 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:05.863240 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0810 14:38:05.863462 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23491,"thread_start_us":23674,"threads_started":1}
I20260810 14:38:05.863658 28949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed89de3ebf5848068a7e2cd893d74697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:05.866812 28732 tablet_service.cc:1467] Processing CreateTablet for tablet ed89de3ebf5848068a7e2cd893d74697 (DEFAULT_TABLE table=test-table [id=8cc0f9db8d7e48a598fc23b3e2fe7bf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:05.866712 28501 tablet_service.cc:1467] Processing CreateTablet for tablet ed89de3ebf5848068a7e2cd893d74697 (DEFAULT_TABLE table=test-table [id=8cc0f9db8d7e48a598fc23b3e2fe7bf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:05.867961 28732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed89de3ebf5848068a7e2cd893d74697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:05.868036 28501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed89de3ebf5848068a7e2cd893d74697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:05.879290 29092 tablet_bootstrap.cc:492] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: Bootstrap starting.
I20260810 14:38:05.883203 29095 tablet_bootstrap.cc:492] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1: Bootstrap starting.
I20260810 14:38:05.883607 29092 tablet_bootstrap.cc:654] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:05.884850 29093 tablet_bootstrap.cc:492] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: Bootstrap starting.
I20260810 14:38:05.885068 29092 log.cc:824] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:05.887553 29095 tablet_bootstrap.cc:654] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:05.888386 29093 tablet_bootstrap.cc:654] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:05.888588 29092 tablet_bootstrap.cc:492] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: No bootstrap required, opened a new log
I20260810 14:38:05.888890 29092 ts_tablet_manager.cc:1397] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260810 14:38:05.888957 29095 log.cc:824] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:05.890069 29093 log.cc:824] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:05.892107 29095 tablet_bootstrap.cc:492] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1: No bootstrap required, opened a new log
I20260810 14:38:05.892381 29095 ts_tablet_manager.cc:1397] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 14:38:05.892575 29093 tablet_bootstrap.cc:492] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: No bootstrap required, opened a new log
I20260810 14:38:05.892812 29093 ts_tablet_manager.cc:1397] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 14:38:05.900104 29092 raft_consensus.cc:348] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f46caf9f72e41a9aa57a466222357a8" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36005 } } peers { permanent_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37153 } } peers { permanent_uuid: "cb6de593a0ac4723a9da7b87f2136847" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 32791 } }
I20260810 14:38:05.900327 29093 raft_consensus.cc:348] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f46caf9f72e41a9aa57a466222357a8" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36005 } } peers { permanent_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37153 } } peers { permanent_uuid: "cb6de593a0ac4723a9da7b87f2136847" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 32791 } }
I20260810 14:38:05.900596 29092 raft_consensus.cc:374] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:05.900732 29093 raft_consensus.cc:374] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:05.900753 29092 raft_consensus.cc:729] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6de593a0ac4723a9da7b87f2136847, State: Initialized, Role: FOLLOWER
I20260810 14:38:05.900876 29093 raft_consensus.cc:729] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f46caf9f72e41a9aa57a466222357a8, State: Initialized, Role: FOLLOWER
I20260810 14:38:05.901470 29093 consensus_queue.cc:260] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f46caf9f72e41a9aa57a466222357a8" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36005 } } peers { permanent_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37153 } } peers { permanent_uuid: "cb6de593a0ac4723a9da7b87f2136847" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 32791 } }
I20260810 14:38:05.901410 29092 consensus_queue.cc:260] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f46caf9f72e41a9aa57a466222357a8" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36005 } } peers { permanent_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37153 } } peers { permanent_uuid: "cb6de593a0ac4723a9da7b87f2136847" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 32791 } }
I20260810 14:38:05.902590 29056 heartbeater.cc:503] Master 127.21.55.126:41231 was elected leader, sending a full tablet report...
I20260810 14:38:05.903210 29093 ts_tablet_manager.cc:1428] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: Time spent starting tablet: real 0.010s user 0.006s sys 0.005s
I20260810 14:38:05.904213 28842 heartbeater.cc:503] Master 127.21.55.126:41231 was elected leader, sending a full tablet report...
I20260810 14:38:05.905224 29092 ts_tablet_manager.cc:1428] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: Time spent starting tablet: real 0.016s user 0.009s sys 0.008s
I20260810 14:38:05.905643 29095 raft_consensus.cc:348] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f46caf9f72e41a9aa57a466222357a8" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36005 } } peers { permanent_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37153 } } peers { permanent_uuid: "cb6de593a0ac4723a9da7b87f2136847" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 32791 } }
I20260810 14:38:05.906116 29095 raft_consensus.cc:374] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:05.906286 29095 raft_consensus.cc:729] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9eba0550f3b042299ca21d7f8c52f9c1, State: Initialized, Role: FOLLOWER
I20260810 14:38:05.907150 29095 consensus_queue.cc:260] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f46caf9f72e41a9aa57a466222357a8" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36005 } } peers { permanent_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37153 } } peers { permanent_uuid: "cb6de593a0ac4723a9da7b87f2136847" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 32791 } }
I20260810 14:38:05.909216 29095 ts_tablet_manager.cc:1428] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
W20260810 14:38:05.926221 28843 tablet.cc:2369] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:38:05.997490 29057 tablet.cc:2369] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:38:06.008080 29101 raft_consensus.cc:482] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:38:06.008519 29101 raft_consensus.cc:504] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f46caf9f72e41a9aa57a466222357a8" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36005 } } peers { permanent_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37153 } } peers { permanent_uuid: "cb6de593a0ac4723a9da7b87f2136847" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 32791 } }
I20260810 14:38:06.010561 29101 leader_election.cc:288] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f46caf9f72e41a9aa57a466222357a8 (127.21.55.67:36005), cb6de593a0ac4723a9da7b87f2136847 (127.21.55.66:32791)
I20260810 14:38:06.020123 29090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:06.011144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:58449 (local address 127.21.55.66:32791)
0810 14:38:06.011345 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:06.011352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:06.012112 (+ 760us) server_negotiation.cc:407] Connection header received
0810 14:38:06.012246 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:06.012252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:06.012329 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:06.012429 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:06.013308 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.014441 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.017714 (+ 3273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.018997 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.019174 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:06.019250 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:06.019723 (+ 473us) server_negotiation.cc:299] Negotiation successful
0810 14:38:06.019851 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:38:06.020123 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:06.011146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:39101 (local address 127.21.55.67:36005)
0810 14:38:06.011345 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:06.011352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:06.012044 (+ 692us) server_negotiation.cc:407] Connection header received
0810 14:38:06.012246 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:06.012252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:06.012329 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:06.012428 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:06.013308 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.014459 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.017714 (+ 3255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.019012 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.019174 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:06.019250 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:06.019723 (+ 473us) server_negotiation.cc:299] Negotiation successful
0810 14:38:06.019849 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:38:06.020378 29102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:06.010921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:36005 (local address 127.21.55.65:39101)
0810 14:38:06.011913 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:06.011934 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:06.012106 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:06.012462 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:06.012467 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:06.012504 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:06.013119 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:06.013130 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.014608 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.014613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:06.017516 (+ 2903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:06.017533 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.019291 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.019300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:06.019541 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:06.019574 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:38:06.019681 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:38:06.019878 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":250,"thread_start_us":598,"threads_started":1}
I20260810 14:38:06.020408 29103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:06.010986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:32791 (local address 127.21.55.65:58449)
0810 14:38:06.011996 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:06.012012 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:06.012108 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:06.012462 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:06.012474 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:06.012526 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:06.013090 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:06.013110 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.014610 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.014614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:06.017516 (+ 2902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:06.017533 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.019308 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.019313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:06.019534 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:06.019575 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 14:38:06.019681 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 14:38:06.019878 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"mutex_wait_us":330,"spinlock_wait_cycles":39680,"thread_start_us":581,"threads_started":1}
I20260810 14:38:06.021802 28990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed89de3ebf5848068a7e2cd893d74697" candidate_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f46caf9f72e41a9aa57a466222357a8" is_pre_election: true
I20260810 14:38:06.021802 28770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed89de3ebf5848068a7e2cd893d74697" candidate_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6de593a0ac4723a9da7b87f2136847" is_pre_election: true
I20260810 14:38:06.022382 28990 raft_consensus.cc:2454] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eba0550f3b042299ca21d7f8c52f9c1 in term 0.
I20260810 14:38:06.022382 28770 raft_consensus.cc:2454] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9eba0550f3b042299ca21d7f8c52f9c1 in term 0.
I20260810 14:38:06.023232 28431 leader_election.cc:302] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eba0550f3b042299ca21d7f8c52f9c1, cb6de593a0ac4723a9da7b87f2136847; no voters:
I20260810 14:38:06.023588 29101 raft_consensus.cc:2781] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:38:06.023736 29101 raft_consensus.cc:482] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:38:06.023891 29101 raft_consensus.cc:3037] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:06.026161 29101 raft_consensus.cc:504] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f46caf9f72e41a9aa57a466222357a8" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36005 } } peers { permanent_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37153 } } peers { permanent_uuid: "cb6de593a0ac4723a9da7b87f2136847" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 32791 } }
I20260810 14:38:06.026942 29101 leader_election.cc:288] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [CANDIDATE]: Term 1 election: Requested vote from peers 9f46caf9f72e41a9aa57a466222357a8 (127.21.55.67:36005), cb6de593a0ac4723a9da7b87f2136847 (127.21.55.66:32791)
I20260810 14:38:06.027340 28990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed89de3ebf5848068a7e2cd893d74697" candidate_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f46caf9f72e41a9aa57a466222357a8"
I20260810 14:38:06.027405 28770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed89de3ebf5848068a7e2cd893d74697" candidate_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6de593a0ac4723a9da7b87f2136847"
I20260810 14:38:06.027582 28770 raft_consensus.cc:3037] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:06.027571 28990 raft_consensus.cc:3037] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:06.030815 28770 raft_consensus.cc:2454] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eba0550f3b042299ca21d7f8c52f9c1 in term 1.
I20260810 14:38:06.030815 28990 raft_consensus.cc:2454] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9eba0550f3b042299ca21d7f8c52f9c1 in term 1.
I20260810 14:38:06.031225 28432 leader_election.cc:302] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9eba0550f3b042299ca21d7f8c52f9c1, 9f46caf9f72e41a9aa57a466222357a8; no voters:
I20260810 14:38:06.031558 29101 raft_consensus.cc:2781] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:06.031973 29101 raft_consensus.cc:686] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 1 LEADER]: Becoming Leader. State: Replica: 9eba0550f3b042299ca21d7f8c52f9c1, State: Running, Role: LEADER
I20260810 14:38:06.032454 29101 consensus_queue.cc:237] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f46caf9f72e41a9aa57a466222357a8" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36005 } } peers { permanent_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37153 } } peers { permanent_uuid: "cb6de593a0ac4723a9da7b87f2136847" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 32791 } }
I20260810 14:38:06.037611 28286 catalog_manager.cc:5495] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9eba0550f3b042299ca21d7f8c52f9c1 (127.21.55.65). New cstate: current_term: 1 leader_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f46caf9f72e41a9aa57a466222357a8" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9eba0550f3b042299ca21d7f8c52f9c1" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 37153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb6de593a0ac4723a9da7b87f2136847" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 32791 } health_report { overall_health: UNKNOWN } } }
I20260810 14:38:06.061239 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:06.055474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59034 (local address 127.21.55.65:37153)
0810 14:38:06.055706 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:06.055712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:06.055869 (+ 157us) server_negotiation.cc:407] Connection header received
0810 14:38:06.056120 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:06.056124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:06.056202 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:06.056358 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:38:06.056733 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.057459 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.059120 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.059529 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.059663 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:06.060143 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:38:06.060631 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:38:06.060783 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:06.060874 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 14:38:06.060963 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:38:06.071133 28770 raft_consensus.cc:1261] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847 [term 1 FOLLOWER]: Refusing update from remote peer 9eba0550f3b042299ca21d7f8c52f9c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:06.071133 28990 raft_consensus.cc:1261] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8 [term 1 FOLLOWER]: Refusing update from remote peer 9eba0550f3b042299ca21d7f8c52f9c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:06.072150 29104 consensus_queue.cc:1035] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb6de593a0ac4723a9da7b87f2136847" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 32791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:06.072904 29101 consensus_queue.cc:1035] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f46caf9f72e41a9aa57a466222357a8" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:06.085314 29112 mvcc.cc:205] Tried to move back new op lower bound from 7316982522136932352 to 7316982521992896512. Current Snapshot: MvccSnapshot[applied={T|T < 7316982522136932352}]
I20260810 14:38:06.086766 29115 mvcc.cc:205] Tried to move back new op lower bound from 7316982522136932352 to 7316982521992896512. Current Snapshot: MvccSnapshot[applied={T|T < 7316982522136932352}]
I20260810 14:38:06.101413 28288 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:06.101760 28288 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:06.106632 28288 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c49828e1499f450085b0d911ef31885e: Sending DeleteTablet for 3 replicas of tablet ed89de3ebf5848068a7e2cd893d74697
I20260810 14:38:06.107851 28949 tablet_service.cc:1514] Processing DeleteTablet for tablet ed89de3ebf5848068a7e2cd893d74697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:06 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:46238
I20260810 14:38:06.107857 28501 tablet_service.cc:1514] Processing DeleteTablet for tablet ed89de3ebf5848068a7e2cd893d74697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:06 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:59022
I20260810 14:38:06.108227 28732 tablet_service.cc:1514] Processing DeleteTablet for tablet ed89de3ebf5848068a7e2cd893d74697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:06 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:39298
I20260810 14:38:06.110709 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 28399
I20260810 14:38:06.119593 29125 tablet_replica.cc:330] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: stopping tablet replica
I20260810 14:38:06.120115 29125 raft_consensus.cc:2229] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:06.120676 29125 raft_consensus.cc:2258] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:06.123283 29125 ts_tablet_manager.cc:1905] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:06.127224 29131 tablet_replica.cc:330] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1: stopping tablet replica
I20260810 14:38:06.127708 29131 raft_consensus.cc:2229] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:38:06.128664 29131 raft_consensus.cc:2258] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:06.132540 29131 ts_tablet_manager.cc:1905] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:06.133221 29133 tablet_replica.cc:330] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: stopping tablet replica
I20260810 14:38:06.133646 29133 raft_consensus.cc:2229] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:06.134157 29133 raft_consensus.cc:2258] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:06.136482 29133 ts_tablet_manager.cc:1905] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:06.139139 29125 ts_tablet_manager.cc:1918] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:06.139317 29125 log.cc:1196] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/ed89de3ebf5848068a7e2cd893d74697
I20260810 14:38:06.139856 29125 ts_tablet_manager.cc:1939] T ed89de3ebf5848068a7e2cd893d74697 P 9f46caf9f72e41a9aa57a466222357a8: Deleting consensus metadata
I20260810 14:38:06.141788 28271 catalog_manager.cc:4841] TS 9f46caf9f72e41a9aa57a466222357a8 (127.21.55.67:36005): tablet ed89de3ebf5848068a7e2cd893d74697 (table test-table [id=8cc0f9db8d7e48a598fc23b3e2fe7bf9]) successfully deleted
I20260810 14:38:06.143512 29133 ts_tablet_manager.cc:1918] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:06.143687 29133 log.cc:1196] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/ed89de3ebf5848068a7e2cd893d74697
Aug 10 14:38:06 dist-test-slave-h6n0 krb5kdc[28222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:38:06.144398 29133 ts_tablet_manager.cc:1939] T ed89de3ebf5848068a7e2cd893d74697 P cb6de593a0ac4723a9da7b87f2136847: Deleting consensus metadata
I20260810 14:38:06.146215 28270 catalog_manager.cc:4841] TS cb6de593a0ac4723a9da7b87f2136847 (127.21.55.66:32791): tablet ed89de3ebf5848068a7e2cd893d74697 (table test-table [id=8cc0f9db8d7e48a598fc23b3e2fe7bf9]) successfully deleted
I20260810 14:38:06.150169 29131 ts_tablet_manager.cc:1918] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:06.150365 29131 log.cc:1196] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/ed89de3ebf5848068a7e2cd893d74697
I20260810 14:38:06.151185 29131 ts_tablet_manager.cc:1939] T ed89de3ebf5848068a7e2cd893d74697 P 9eba0550f3b042299ca21d7f8c52f9c1: Deleting consensus metadata
I20260810 14:38:06.153195 28268 catalog_manager.cc:4841] TS 9eba0550f3b042299ca21d7f8c52f9c1 (127.21.55.65:37153): tablet ed89de3ebf5848068a7e2cd893d74697 (table test-table [id=8cc0f9db8d7e48a598fc23b3e2fe7bf9]) successfully deleted
I20260810 14:38:06.153276 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:06.113476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59036 (local address 127.21.55.65:37153)
0810 14:38:06.114083 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:06.114097 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:06.129647 (+ 15550us) server_negotiation.cc:407] Connection header received
0810 14:38:06.130816 (+ 1169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:06.130824 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:06.130994 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:06.131119 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:06.134076 (+ 2957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.135408 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.138012 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.138821 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.138967 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:06.145430 (+ 6463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:06.145447 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:06.145470 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:06.145534 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:06.149788 (+ 4254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:06.150717 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:06.150727 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:06.150730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:06.150926 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:06.151426 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:06.151434 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:06.151441 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:06.151948 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:06.152349 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:06.152625 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 14:38:06.152878 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401}
I20260810 14:38:06.154017 28609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:06.270331 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 28399
I20260810 14:38:06.298820 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 28626
Aug 10 14:38:06 dist-test-slave-h6n0 krb5kdc[28222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:38:06.315155 29090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:06.299651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39302 (local address 127.21.55.66:32791)
0810 14:38:06.299834 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:06.299838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:06.299857 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:38:06.300045 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:06.300049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:06.300124 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:06.300260 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:06.301603 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.302489 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.304000 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.304639 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.304750 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:06.309250 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:06.309262 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:06.309277 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:06.309309 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:06.312379 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:06.313045 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:06.313053 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:06.313055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:06.313200 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:06.313546 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:06.313550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:06.313551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:06.313959 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:06.314219 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:06.314481 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 14:38:06.314745 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:38:06.315912 28834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:06.407325 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 28626
I20260810 14:38:06.437920 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 28850
Aug 10 14:38:06 dist-test-slave-h6n0 krb5kdc[28222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:38:06.454253 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:06.438797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46246 (local address 127.21.55.67:36005)
0810 14:38:06.438993 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:06.439000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:06.439021 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:38:06.439221 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:06.439227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:06.439306 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:06.439418 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:06.440720 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.441446 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.443233 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:06.443650 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:06.443793 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:06.448514 (+ 4721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:06.448530 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:06.448546 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:06.448578 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:06.451458 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:06.452187 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:06.452195 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:06.452197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:06.452321 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:06.452727 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:06.452732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:06.452734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:06.453078 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:06.453373 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:06.453711 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 14:38:06.453939 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:38:06.454985 29049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:06.551926 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 28850
I20260810 14:38:06.584705 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 28243
I20260810 14:38:06.585934 28338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:06.699481 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 28243
2026-08-10T14:38:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4905 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T14:38:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:38:06Z Disabled control of system clock
I20260810 14:38:06.785547 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:41497
--webserver_interface=127.21.55.126
--webserver_port=0
--builtin_ntp_servers=127.21.55.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:41497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 14:38:07.118109 29146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:07.118391 29146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:07.118544 29146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:07.125764 29146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:07.125874 29146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:07.125913 29146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:07.126008 29146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:07.126096 29146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:38:07.135311 29146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:41497
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:07.136655 29146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:07.138463 29146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:07.151713 29153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:07.151839 29152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:07.151818 29157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:07.152771 29146 server_base.cc:1034] running on GCE node
I20260810 14:38:07.153530 29146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:07.154994 29146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:07.156225 29146 hybrid_clock.cc:648] HybridClock initialized: now 1786372687156181 us; error 32 us; skew 500 ppm
I20260810 14:38:07.156564 29146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:38:07.159293 29146 webserver.cc:460] Webserver started at http://127.21.55.126:33971/ using document root <none> and password file <none>
I20260810 14:38:07.160550 29146 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:07.160658 29146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:07.160949 29146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:07.163932 29146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "14deb85ea59946b7982dd8ada9b67ac6"
format_stamp: "Formatted at 2026-08-10 14:38:07 on dist-test-slave-h6n0"
server_key: "87efd87c78d57df0b37266c340e07d5d"
server_key_iv: "4970f907a9c9e2b058b2bf44c0d3171b"
server_key_version: "encryptionkey@0"
I20260810 14:38:07.164693 29146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "14deb85ea59946b7982dd8ada9b67ac6"
format_stamp: "Formatted at 2026-08-10 14:38:07 on dist-test-slave-h6n0"
server_key: "87efd87c78d57df0b37266c340e07d5d"
server_key_iv: "4970f907a9c9e2b058b2bf44c0d3171b"
server_key_version: "encryptionkey@0"
I20260810 14:38:07.169826 29146 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260810 14:38:07.173662 29167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:07.175155 29146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:38:07.175298 29146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "14deb85ea59946b7982dd8ada9b67ac6"
format_stamp: "Formatted at 2026-08-10 14:38:07 on dist-test-slave-h6n0"
server_key: "87efd87c78d57df0b37266c340e07d5d"
server_key_iv: "4970f907a9c9e2b058b2bf44c0d3171b"
server_key_version: "encryptionkey@0"
I20260810 14:38:07.175480 29146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:07.181319 29146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:07.182118 29146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:07.182360 29146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:07.202316 29146 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:41497
I20260810 14:38:07.202409 29251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:41497 every 8 connection(s)
I20260810 14:38:07.203739 29146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:38:07.205684 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 29146
I20260810 14:38:07.205916 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:38:07.206895 21725 external_mini_cluster.cc:1428] Setting key adc5f25652ff57da99584ce96aca5777
I20260810 14:38:07.209731 29252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:07.215682 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:07.209020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54144 (local address 127.21.55.126:41497)
0810 14:38:07.209979 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:07.209991 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:07.210036 (+ 45us) server_negotiation.cc:407] Connection header received
0810 14:38:07.211689 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:07.211710 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:07.212095 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:07.212496 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:07.213827 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:07.213833 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:07.213850 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:07.213865 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:07.213964 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:07.213983 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:07.214232 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:07.214380 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 14:38:07.214719 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":299,"threads_started":1}
I20260810 14:38:07.225091 29252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6: Bootstrap starting.
I20260810 14:38:07.229737 29252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:07.231223 29252 log.cc:824] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:07.235051 29252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6: No bootstrap required, opened a new log
I20260810 14:38:07.244256 29252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14deb85ea59946b7982dd8ada9b67ac6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41497 } }
I20260810 14:38:07.244685 29252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:07.244771 29252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14deb85ea59946b7982dd8ada9b67ac6, State: Initialized, Role: FOLLOWER
I20260810 14:38:07.245337 29252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14deb85ea59946b7982dd8ada9b67ac6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41497 } }
I20260810 14:38:07.245571 29252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:38:07.245683 29252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:38:07.245824 29252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:07.248606 29252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14deb85ea59946b7982dd8ada9b67ac6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41497 } }
I20260810 14:38:07.249125 29252 leader_election.cc:302] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14deb85ea59946b7982dd8ada9b67ac6; no voters:
I20260810 14:38:07.249722 29252 leader_election.cc:288] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:38:07.249914 29257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:07.250741 29257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [term 1 LEADER]: Becoming Leader. State: Replica: 14deb85ea59946b7982dd8ada9b67ac6, State: Running, Role: LEADER
I20260810 14:38:07.251372 29257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14deb85ea59946b7982dd8ada9b67ac6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41497 } }
I20260810 14:38:07.252096 29252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:07.255044 29262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14deb85ea59946b7982dd8ada9b67ac6. Latest consensus state: current_term: 1 leader_uuid: "14deb85ea59946b7982dd8ada9b67ac6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14deb85ea59946b7982dd8ada9b67ac6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41497 } } }
I20260810 14:38:07.255541 29262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:07.255867 29258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14deb85ea59946b7982dd8ada9b67ac6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14deb85ea59946b7982dd8ada9b67ac6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41497 } } }
I20260810 14:38:07.256335 29258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:07.257767 29273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:07.262902 29273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:07.273362 29273 catalog_manager.cc:1347] Generated new cluster ID: dca9649771244b668e00f9a33bbe3cb6
I20260810 14:38:07.273597 29273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:07.297394 29273 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:38:07.297904 29273 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:07.313422 29273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6: Generated new TSK 0
I20260810 14:38:07.314339 29273 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:38:07.324389 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41497
--builtin_ntp_servers=127.21.55.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 14:38:07.702451 29288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:07.702744 29288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:07.702847 29288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:07.711621 29288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:07.711753 29288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:07.711884 29288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:38:07.721917 29288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:07.723217 29288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:07.724984 29288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:07.739770 29296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:07.740052 29297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:07.740115 29302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:07.741523 29288 server_base.cc:1034] running on GCE node
I20260810 14:38:07.742250 29288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:07.743402 29288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:07.744648 29288 hybrid_clock.cc:648] HybridClock initialized: now 1786372687744611 us; error 57 us; skew 500 ppm
I20260810 14:38:07.745018 29288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:38:07.747677 29288 webserver.cc:460] Webserver started at http://127.21.55.65:45077/ using document root <none> and password file <none>
I20260810 14:38:07.748401 29288 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:07.748499 29288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:07.748838 29288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:07.751811 29288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "80aefed4f78f4d2fabf8925aa34f19dd"
format_stamp: "Formatted at 2026-08-10 14:38:07 on dist-test-slave-h6n0"
server_key: "175ae21b0614248d59445e979b8d98d8"
server_key_iv: "f144744c5222608b565b87f09510d460"
server_key_version: "encryptionkey@0"
I20260810 14:38:07.752669 29288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "80aefed4f78f4d2fabf8925aa34f19dd"
format_stamp: "Formatted at 2026-08-10 14:38:07 on dist-test-slave-h6n0"
server_key: "175ae21b0614248d59445e979b8d98d8"
server_key_iv: "f144744c5222608b565b87f09510d460"
server_key_version: "encryptionkey@0"
I20260810 14:38:07.758951 29288 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 14:38:07.762885 29309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:07.764297 29288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:38:07.764480 29288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "80aefed4f78f4d2fabf8925aa34f19dd"
format_stamp: "Formatted at 2026-08-10 14:38:07 on dist-test-slave-h6n0"
server_key: "175ae21b0614248d59445e979b8d98d8"
server_key_iv: "f144744c5222608b565b87f09510d460"
server_key_version: "encryptionkey@0"
I20260810 14:38:07.764720 29288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:07.770501 29288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:07.771363 29288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:07.771613 29288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:07.772576 29288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:07.774004 29288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:07.774099 29288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:07.774178 29288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:07.774236 29288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:07.813261 29288 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:43615
I20260810 14:38:07.813380 29510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:43615 every 8 connection(s)
I20260810 14:38:07.814847 29288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:38:07.823366 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 29288
I20260810 14:38:07.823393 29514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:07.817686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:60171 (local address 127.21.55.126:41497)
0810 14:38:07.818097 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:07.818102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:07.819328 (+ 1226us) server_negotiation.cc:407] Connection header received
0810 14:38:07.821054 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:07.821059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:07.821132 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:07.821253 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:07.822597 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:07.822603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:07.822608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:07.822625 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:07.822675 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:07.822687 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:07.822823 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:07.823020 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 14:38:07.823091 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":215,"threads_started":1}
I20260810 14:38:07.823588 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:38:07.824262 21725 external_mini_cluster.cc:1428] Setting key 3d70c8312c3e0ea7736e74bdb1a7b2f2
I20260810 14:38:07.824270 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:07.818273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:41497 (local address 127.21.55.65:60171)
0810 14:38:07.819174 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:07.819215 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:07.820772 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:07.821478 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:07.821486 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:07.821567 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:07.822283 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:38:07.822286 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:07.822322 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:38:07.822403 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:07.822789 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:07.822802 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:07.822810 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 14:38:07.823104 (+ 294us) client_negotiation.cc:240] Negotiation successful
0810 14:38:07.823333 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":319,"threads_started":1}
I20260810 14:38:07.825977 29511 heartbeater.cc:347] Connected to a master server at 127.21.55.126:41497
I20260810 14:38:07.826452 29511 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:07.827232 29511 heartbeater.cc:511] Master 127.21.55.126:41497 requested a full tablet report, sending...
I20260810 14:38:07.827971 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41497
--builtin_ntp_servers=127.21.55.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 14:38:07.829337 29195 ts_manager.cc:194] Registered new tserver with Master: 80aefed4f78f4d2fabf8925aa34f19dd (127.21.55.65:43615)
W20260810 14:38:08.203248 29518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:08.203579 29518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:08.203755 29518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:08.214300 29518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:08.214489 29518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:08.214716 29518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:38:08.229671 29518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:08.231184 29518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:08.233055 29518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:08.250856 29524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:08.250821 29532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:08.251055 29525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:08.252617 29518 server_base.cc:1034] running on GCE node
I20260810 14:38:08.253618 29518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:08.254951 29518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:08.256228 29518 hybrid_clock.cc:648] HybridClock initialized: now 1786372688256181 us; error 67 us; skew 500 ppm
I20260810 14:38:08.256539 29518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:38:08.259188 29518 webserver.cc:460] Webserver started at http://127.21.55.66:41107/ using document root <none> and password file <none>
I20260810 14:38:08.259931 29518 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:08.260080 29518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:08.260437 29518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:08.263633 29518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "890d7bab8502455abe2e3d2f341be9fc"
format_stamp: "Formatted at 2026-08-10 14:38:08 on dist-test-slave-h6n0"
server_key: "d221e332a6b71910d2c5138f06097da7"
server_key_iv: "b05c5a1842a8b6df0bcd1dd9f154b66a"
server_key_version: "encryptionkey@0"
I20260810 14:38:08.264583 29518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "890d7bab8502455abe2e3d2f341be9fc"
format_stamp: "Formatted at 2026-08-10 14:38:08 on dist-test-slave-h6n0"
server_key: "d221e332a6b71910d2c5138f06097da7"
server_key_iv: "b05c5a1842a8b6df0bcd1dd9f154b66a"
server_key_version: "encryptionkey@0"
I20260810 14:38:08.271812 29518 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.003s
I20260810 14:38:08.277191 29541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:08.279155 29518 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 14:38:08.279345 29518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "890d7bab8502455abe2e3d2f341be9fc"
format_stamp: "Formatted at 2026-08-10 14:38:08 on dist-test-slave-h6n0"
server_key: "d221e332a6b71910d2c5138f06097da7"
server_key_iv: "b05c5a1842a8b6df0bcd1dd9f154b66a"
server_key_version: "encryptionkey@0"
I20260810 14:38:08.279634 29518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:08.286693 29518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:08.287568 29518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:08.287860 29518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:08.288929 29518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:08.290500 29518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:08.290680 29518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:08.290815 29518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:08.290915 29518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:08.336243 29518 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:38743
I20260810 14:38:08.336294 29723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:38743 every 8 connection(s)
I20260810 14:38:08.337644 29518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:38:08.343940 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 29518
I20260810 14:38:08.344264 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:38:08.344781 21725 external_mini_cluster.cc:1428] Setting key f80bc9188c9d333af8ef39a52c23578d
I20260810 14:38:08.348538 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41497
--builtin_ntp_servers=127.21.55.84:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 14:38:08.349417 29732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:08.340511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:43749 (local address 127.21.55.126:41497)
0810 14:38:08.341069 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:08.341076 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:08.342930 (+ 1854us) server_negotiation.cc:407] Connection header received
0810 14:38:08.345651 (+ 2721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:08.345658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:08.345735 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:08.345888 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:08.348092 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:08.348100 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:08.348107 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:08.348131 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:08.348200 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:08.348218 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:08.348384 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:08.348997 (+ 613us) server_negotiation.cc:299] Negotiation successful
0810 14:38:08.349097 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":273,"threads_started":1}
I20260810 14:38:08.350917 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:08.341149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:41497 (local address 127.21.55.66:43749)
0810 14:38:08.342632 (+ 1483us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:08.342717 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:08.345162 (+ 2445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:08.346313 (+ 1151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:08.346326 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:08.346467 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:08.347616 (+ 1149us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:38:08.347619 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:08.347677 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:38:08.347827 (+ 150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:08.348463 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:08.348491 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:08.348501 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:38:08.349134 (+ 633us) client_negotiation.cc:240] Negotiation successful
0810 14:38:08.349480 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":412,"threads_started":1}
I20260810 14:38:08.353214 29724 heartbeater.cc:347] Connected to a master server at 127.21.55.126:41497
I20260810 14:38:08.353638 29724 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:08.354339 29724 heartbeater.cc:511] Master 127.21.55.126:41497 requested a full tablet report, sending...
I20260810 14:38:08.356094 29195 ts_manager.cc:194] Registered new tserver with Master: 890d7bab8502455abe2e3d2f341be9fc (127.21.55.66:38743)
W20260810 14:38:08.725747 29734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:08.726188 29734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:08.726395 29734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:08.735862 29734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:08.736074 29734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:08.736424 29734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:38:08.749495 29734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:41497
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:08.753439 29734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:08.756978 29734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:08.774333 29744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:08.774633 29746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:08.775939 29750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:08.776662 29734 server_base.cc:1034] running on GCE node
I20260810 14:38:08.777499 29734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:08.778846 29734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:08.780241 29734 hybrid_clock.cc:648] HybridClock initialized: now 1786372688780118 us; error 96 us; skew 500 ppm
I20260810 14:38:08.780730 29734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:38:08.784109 29734 webserver.cc:460] Webserver started at http://127.21.55.67:40987/ using document root <none> and password file <none>
I20260810 14:38:08.785143 29734 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:08.785588 29734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:08.786461 29734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:08.791405 29734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "523ad1c25b744f43b92b2316066ef044"
format_stamp: "Formatted at 2026-08-10 14:38:08 on dist-test-slave-h6n0"
server_key: "a6f54a9fcba4c765190dbe85e1522e40"
server_key_iv: "7001796f4ffa5da69f8648bc13daa3d4"
server_key_version: "encryptionkey@0"
I20260810 14:38:08.792590 29734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "523ad1c25b744f43b92b2316066ef044"
format_stamp: "Formatted at 2026-08-10 14:38:08 on dist-test-slave-h6n0"
server_key: "a6f54a9fcba4c765190dbe85e1522e40"
server_key_iv: "7001796f4ffa5da69f8648bc13daa3d4"
server_key_version: "encryptionkey@0"
I20260810 14:38:08.800976 29734 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.008s
I20260810 14:38:08.806429 29760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:08.808276 29734 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:38:08.808494 29734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "523ad1c25b744f43b92b2316066ef044"
format_stamp: "Formatted at 2026-08-10 14:38:08 on dist-test-slave-h6n0"
server_key: "a6f54a9fcba4c765190dbe85e1522e40"
server_key_iv: "7001796f4ffa5da69f8648bc13daa3d4"
server_key_version: "encryptionkey@0"
I20260810 14:38:08.808781 29734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:08.818241 29734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:08.819399 29734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:08.819769 29734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:08.821316 29734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:08.823840 29734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:08.823987 29734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:08.824122 29734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:08.824232 29734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:08.832459 29511 heartbeater.cc:503] Master 127.21.55.126:41497 was elected leader, sending a full tablet report...
I20260810 14:38:08.881250 29734 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:34283
I20260810 14:38:08.881467 29948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:34283 every 8 connection(s)
I20260810 14:38:08.882838 29734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:38:08.888209 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 29734
I20260810 14:38:08.888403 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:38:08.888865 21725 external_mini_cluster.cc:1428] Setting key 8cdf60b5e18eed4f332794afcb78046a
I20260810 14:38:08.894320 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:08.886212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:38011 (local address 127.21.55.126:41497)
0810 14:38:08.886779 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:08.886788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:08.889394 (+ 2606us) server_negotiation.cc:407] Connection header received
0810 14:38:08.891352 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:08.891358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:08.891423 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:08.891538 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:08.893482 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:08.893486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:08.893490 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:08.893506 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:08.893552 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:08.893564 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:08.893668 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:08.893981 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 14:38:08.894048 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":305,"threads_started":1}
I20260810 14:38:08.895687 29956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:08.886825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:41497 (local address 127.21.55.67:38011)
0810 14:38:08.888186 (+ 1361us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:08.888243 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:08.890988 (+ 2745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:08.891854 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:08.891871 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:08.892015 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:08.893019 (+ 1004us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:38:08.893025 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:08.893091 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:38:08.893268 (+ 177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:08.893683 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:08.893697 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:08.893707 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:38:08.894043 (+ 336us) client_negotiation.cc:240] Negotiation successful
0810 14:38:08.894353 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":417,"threads_started":1}
I20260810 14:38:08.897194 29949 heartbeater.cc:347] Connected to a master server at 127.21.55.126:41497
I20260810 14:38:08.897539 29949 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:08.898363 29949 heartbeater.cc:511] Master 127.21.55.126:41497 requested a full tablet report, sending...
I20260810 14:38:08.900168 29195 ts_manager.cc:194] Registered new tserver with Master: 523ad1c25b744f43b92b2316066ef044 (127.21.55.67:34283)
I20260810 14:38:08.904152 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:38:08.912015 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:08.909016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54158 (local address 127.21.55.126:41497)
0810 14:38:08.909186 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:08.909191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:08.909838 (+ 647us) server_negotiation.cc:407] Connection header received
0810 14:38:08.909963 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:08.909971 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:08.910071 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:08.910209 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:08.911246 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:08.911255 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:08.911262 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:08.911286 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:08.911336 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:08.911357 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:08.911520 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:08.911579 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 14:38:08.911652 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 14:38:08.916543 29195 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:38:08.923398 29195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:38:08.954094 29974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:08.944438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:34283 (local address 127.0.0.1:47166)
0810 14:38:08.945246 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:08.945304 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:08.945460 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:08.948435 (+ 2975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:08.948443 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:08.948533 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:08.951133 (+ 2600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:38:08.951140 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:08.951164 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:38:08.952452 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:08.953047 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:08.953064 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:08.953076 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 14:38:08.953365 (+ 289us) client_negotiation.cc:240] Negotiation successful
0810 14:38:08.953485 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":32,"thread_start_us":530,"threads_started":1}
I20260810 14:38:08.954500 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:08.946137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.21.55.66:38743)
0810 14:38:08.948123 (+ 1986us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:08.948131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:08.948159 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:38:08.948231 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:08.948238 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:08.948482 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:08.948710 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:08.952718 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:08.952733 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:08.952752 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:08.952802 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:08.952884 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:08.952916 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:08.953768 (+ 852us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:08.953886 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 14:38:08.954018 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1808,"thread_start_us":1680,"threads_started":1}
I20260810 14:38:08.955389 29976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:08.945765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:38743 (local address 127.0.0.1:39850)
0810 14:38:08.946649 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:08.946665 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:08.946749 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:08.948748 (+ 1999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:08.948756 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:08.948799 (+ 43us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:08.950909 (+ 2110us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:38:08.950915 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:08.950958 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:38:08.952458 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:08.953079 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:08.953088 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:08.953098 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:38:08.953307 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 14:38:08.953461 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":531,"threads_started":1}
I20260810 14:38:08.956907 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:08.944756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47166 (local address 127.21.55.67:34283)
0810 14:38:08.945607 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:08.945616 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:08.945657 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:38:08.945761 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:08.945770 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:08.948201 (+ 2431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:08.949508 (+ 1307us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:08.952711 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:08.952725 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:08.952745 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:08.952772 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:08.952851 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:08.952878 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:08.956199 (+ 3321us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:08.956282 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 14:38:08.956394 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":444,"threads_started":1}
I20260810 14:38:08.961427 29835 tablet_service.cc:1467] Processing CreateTablet for tablet 467f127ab67943d9a7c3f4a436c5740c (DEFAULT_TABLE table=test-table [id=23982ed195024a30844689f4b5af942f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:08.961505 29616 tablet_service.cc:1467] Processing CreateTablet for tablet 467f127ab67943d9a7c3f4a436c5740c (DEFAULT_TABLE table=test-table [id=23982ed195024a30844689f4b5af942f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:08.963060 29835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467f127ab67943d9a7c3f4a436c5740c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:08.963066 29616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467f127ab67943d9a7c3f4a436c5740c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:08.963269 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:08.947034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:43615 (local address 127.0.0.1:39870)
0810 14:38:08.947755 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:08.947773 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:08.947888 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:08.959796 (+ 11908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:08.959802 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:08.959830 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:08.961949 (+ 2119us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:38:08.961953 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:08.961973 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:38:08.962024 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:08.962618 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:08.962624 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:08.962629 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:38:08.962766 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 14:38:08.962820 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":407,"threads_started":1}
I20260810 14:38:08.963954 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:08.947290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39870 (local address 127.21.55.65:43615)
0810 14:38:08.959159 (+ 11869us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:08.959167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:08.959199 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:38:08.959276 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:08.959284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:08.959602 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:08.959772 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:08.962205 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:08.962215 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:08.962228 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:08.962252 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:08.962315 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:08.962336 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:08.962519 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:08.963495 (+ 976us) server_negotiation.cc:299] Negotiation successful
0810 14:38:08.963598 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11680,"thread_start_us":11462,"threads_started":1}
I20260810 14:38:08.967427 29400 tablet_service.cc:1467] Processing CreateTablet for tablet 467f127ab67943d9a7c3f4a436c5740c (DEFAULT_TABLE table=test-table [id=23982ed195024a30844689f4b5af942f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:08.968907 29400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 467f127ab67943d9a7c3f4a436c5740c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:08.982599 29985 tablet_bootstrap.cc:492] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: Bootstrap starting.
I20260810 14:38:08.984876 29986 tablet_bootstrap.cc:492] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044: Bootstrap starting.
I20260810 14:38:08.986835 29985 tablet_bootstrap.cc:654] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:08.987658 29987 tablet_bootstrap.cc:492] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd: Bootstrap starting.
I20260810 14:38:08.988422 29985 log.cc:824] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:08.989063 29986 tablet_bootstrap.cc:654] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:08.990566 29986 log.cc:824] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:08.992404 29987 tablet_bootstrap.cc:654] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:08.993883 29985 tablet_bootstrap.cc:492] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: No bootstrap required, opened a new log
I20260810 14:38:08.994356 29985 ts_tablet_manager.cc:1397] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260810 14:38:08.994903 29986 tablet_bootstrap.cc:492] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044: No bootstrap required, opened a new log
I20260810 14:38:08.995254 29986 ts_tablet_manager.cc:1397] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260810 14:38:08.996285 29987 log.cc:824] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:08.999992 29987 tablet_bootstrap.cc:492] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd: No bootstrap required, opened a new log
I20260810 14:38:09.000304 29987 ts_tablet_manager.cc:1397] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260810 14:38:09.006428 29986 raft_consensus.cc:348] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } }
I20260810 14:38:09.006999 29986 raft_consensus.cc:374] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:09.007136 29986 raft_consensus.cc:729] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 523ad1c25b744f43b92b2316066ef044, State: Initialized, Role: FOLLOWER
I20260810 14:38:09.007478 29985 raft_consensus.cc:348] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } }
I20260810 14:38:09.007987 29985 raft_consensus.cc:374] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:09.008139 29985 raft_consensus.cc:729] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 890d7bab8502455abe2e3d2f341be9fc, State: Initialized, Role: FOLLOWER
I20260810 14:38:09.008052 29986 consensus_queue.cc:260] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } }
I20260810 14:38:09.008813 29985 consensus_queue.cc:260] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } }
I20260810 14:38:09.009676 29949 heartbeater.cc:503] Master 127.21.55.126:41497 was elected leader, sending a full tablet report...
I20260810 14:38:09.010494 29986 ts_tablet_manager.cc:1428] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260810 14:38:09.014448 29724 heartbeater.cc:503] Master 127.21.55.126:41497 was elected leader, sending a full tablet report...
I20260810 14:38:09.015184 29985 ts_tablet_manager.cc:1428] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: Time spent starting tablet: real 0.021s user 0.018s sys 0.005s
I20260810 14:38:09.016840 29987 raft_consensus.cc:348] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } }
I20260810 14:38:09.017275 29987 raft_consensus.cc:374] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:09.017385 29987 raft_consensus.cc:729] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80aefed4f78f4d2fabf8925aa34f19dd, State: Initialized, Role: FOLLOWER
I20260810 14:38:09.018049 29987 consensus_queue.cc:260] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } }
I20260810 14:38:09.019739 29987 ts_tablet_manager.cc:1428] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd: Time spent starting tablet: real 0.019s user 0.008s sys 0.007s
I20260810 14:38:09.031123 29995 raft_consensus.cc:482] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:38:09.031466 29993 raft_consensus.cc:482] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:38:09.031422 29995 raft_consensus.cc:504] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } }
I20260810 14:38:09.031791 29993 raft_consensus.cc:504] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } }
I20260810 14:38:09.033209 29995 leader_election.cc:288] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 523ad1c25b744f43b92b2316066ef044 (127.21.55.67:34283), 80aefed4f78f4d2fabf8925aa34f19dd (127.21.55.65:43615)
I20260810 14:38:09.035567 29993 leader_election.cc:288] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 890d7bab8502455abe2e3d2f341be9fc (127.21.55.66:38743), 80aefed4f78f4d2fabf8925aa34f19dd (127.21.55.65:43615)
I20260810 14:38:09.037925 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.033758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:38215 (local address 127.21.55.67:34283)
0810 14:38:09.033953 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:09.033962 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:09.034731 (+ 769us) server_negotiation.cc:407] Connection header received
0810 14:38:09.035534 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:09.035544 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:09.035612 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:09.035696 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:09.037058 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:09.037067 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:09.037076 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:09.037100 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:09.037154 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:09.037178 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:09.037320 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:09.037546 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 14:38:09.037608 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 14:38:09.038932 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.033646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:34283 (local address 127.21.55.66:38215)
0810 14:38:09.034633 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.034652 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:09.035379 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:09.035924 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:09.035928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:09.035953 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:09.036828 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:38:09.036834 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:09.036857 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:38:09.036912 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:09.037379 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:09.037384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:09.037388 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:38:09.038345 (+ 957us) client_negotiation.cc:240] Negotiation successful
0810 14:38:09.038409 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":544,"threads_started":1}
I20260810 14:38:09.039127 29956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.035655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:38743 (local address 127.21.55.67:58905)
0810 14:38:09.036071 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.036094 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:09.036190 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:09.037060 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:09.037072 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:09.037104 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:09.038198 (+ 1094us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:38:09.038203 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:09.038226 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:38:09.038285 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:09.038681 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:09.038686 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:09.038692 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:38:09.038798 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 14:38:09.038866 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284}
I20260810 14:38:09.039198 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.036416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:58905 (local address 127.21.55.66:38743)
0810 14:38:09.036586 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:09.036595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:09.036621 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:38:09.036678 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:09.036684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:09.036741 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:09.036861 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:09.038433 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:09.038440 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:09.038445 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:09.038470 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:09.038512 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:09.038564 (+ 52us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:09.038704 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:09.038819 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 14:38:09.038876 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 14:38:09.040356 29867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467f127ab67943d9a7c3f4a436c5740c" candidate_uuid: "890d7bab8502455abe2e3d2f341be9fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "523ad1c25b744f43b92b2316066ef044" is_pre_election: true
I20260810 14:38:09.040616 29642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467f127ab67943d9a7c3f4a436c5740c" candidate_uuid: "523ad1c25b744f43b92b2316066ef044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "890d7bab8502455abe2e3d2f341be9fc" is_pre_election: true
I20260810 14:38:09.040829 29867 raft_consensus.cc:2454] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 890d7bab8502455abe2e3d2f341be9fc in term 0.
I20260810 14:38:09.040930 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.034431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:53111 (local address 127.21.55.65:43615)
0810 14:38:09.034642 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:09.034651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:09.035219 (+ 568us) server_negotiation.cc:407] Connection header received
0810 14:38:09.035948 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:09.035953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:09.036128 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:09.036231 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:09.040151 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:09.040156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:09.040160 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:09.040180 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:09.040225 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:09.040243 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:09.040342 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:09.040604 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 14:38:09.040667 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 14:38:09.041091 29642 raft_consensus.cc:2454] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 523ad1c25b744f43b92b2316066ef044 in term 0.
I20260810 14:38:09.041501 29548 leader_election.cc:302] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 523ad1c25b744f43b92b2316066ef044, 890d7bab8502455abe2e3d2f341be9fc; no voters:
I20260810 14:38:09.042032 29995 raft_consensus.cc:2781] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:38:09.042187 29995 raft_consensus.cc:482] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:38:09.042307 29995 raft_consensus.cc:3037] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:09.042263 29765 leader_election.cc:302] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 523ad1c25b744f43b92b2316066ef044, 890d7bab8502455abe2e3d2f341be9fc; no voters:
I20260810 14:38:09.042797 29993 raft_consensus.cc:2781] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:38:09.042985 29993 raft_consensus.cc:482] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:38:09.043123 29993 raft_consensus.cc:3037] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:09.043166 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.036714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:43615 (local address 127.21.55.67:49369)
0810 14:38:09.037509 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.037525 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:09.037605 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:09.039086 (+ 1481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:09.039090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:09.039116 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:09.041525 (+ 2409us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:38:09.041530 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:09.041564 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:38:09.041622 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:09.042078 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:09.042083 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:09.042089 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:38:09.042335 (+ 246us) client_negotiation.cc:240] Negotiation successful
0810 14:38:09.042507 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":1014,"threads_started":1}
I20260810 14:38:09.045018 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.037542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:49369 (local address 127.21.55.65:43615)
0810 14:38:09.038731 (+ 1189us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:09.038738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:09.038763 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:38:09.038817 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:09.038824 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:09.038876 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:09.039489 (+ 613us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:09.041857 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:09.041864 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:09.041869 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:09.041886 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:09.041924 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:09.041941 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:09.044261 (+ 2320us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:09.044363 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 14:38:09.044463 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":914,"thread_start_us":842,"threads_started":1}
I20260810 14:38:09.046933 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.033722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:43615 (local address 127.21.55.66:53111)
0810 14:38:09.035091 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.035110 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:09.035238 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:09.036443 (+ 1205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:09.036451 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:09.036475 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:09.039906 (+ 3431us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:38:09.039909 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:09.039926 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:38:09.039971 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:09.040440 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:09.040444 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:09.040449 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:38:09.041418 (+ 969us) client_negotiation.cc:240] Negotiation successful
0810 14:38:09.041471 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1229,"thread_start_us":1032,"threads_started":1}
I20260810 14:38:09.046900 29436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467f127ab67943d9a7c3f4a436c5740c" candidate_uuid: "523ad1c25b744f43b92b2316066ef044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" is_pre_election: true
I20260810 14:38:09.047456 29436 raft_consensus.cc:2454] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 523ad1c25b744f43b92b2316066ef044 in term 0.
I20260810 14:38:09.046810 29995 raft_consensus.cc:504] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } }
I20260810 14:38:09.048373 29993 raft_consensus.cc:504] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } }
I20260810 14:38:09.048532 29995 leader_election.cc:288] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [CANDIDATE]: Term 1 election: Requested vote from peers 523ad1c25b744f43b92b2316066ef044 (127.21.55.67:34283), 80aefed4f78f4d2fabf8925aa34f19dd (127.21.55.65:43615)
I20260810 14:38:09.048910 29433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467f127ab67943d9a7c3f4a436c5740c" candidate_uuid: "890d7bab8502455abe2e3d2f341be9fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80aefed4f78f4d2fabf8925aa34f19dd"
I20260810 14:38:09.049193 29433 raft_consensus.cc:3037] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:09.049455 29642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467f127ab67943d9a7c3f4a436c5740c" candidate_uuid: "523ad1c25b744f43b92b2316066ef044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "890d7bab8502455abe2e3d2f341be9fc"
I20260810 14:38:09.049489 29867 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467f127ab67943d9a7c3f4a436c5740c" candidate_uuid: "890d7bab8502455abe2e3d2f341be9fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "523ad1c25b744f43b92b2316066ef044"
I20260810 14:38:09.049759 29642 raft_consensus.cc:2379] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 523ad1c25b744f43b92b2316066ef044 in current term 1: Already voted for candidate 890d7bab8502455abe2e3d2f341be9fc in this term.
I20260810 14:38:09.049818 29867 raft_consensus.cc:2379] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 890d7bab8502455abe2e3d2f341be9fc in current term 1: Already voted for candidate 523ad1c25b744f43b92b2316066ef044 in this term.
I20260810 14:38:09.050815 29993 leader_election.cc:288] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [CANDIDATE]: Term 1 election: Requested vote from peers 890d7bab8502455abe2e3d2f341be9fc (127.21.55.66:38743), 80aefed4f78f4d2fabf8925aa34f19dd (127.21.55.65:43615)
I20260810 14:38:09.050886 29436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467f127ab67943d9a7c3f4a436c5740c" candidate_uuid: "890d7bab8502455abe2e3d2f341be9fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" is_pre_election: true
I20260810 14:38:09.051281 29430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "467f127ab67943d9a7c3f4a436c5740c" candidate_uuid: "523ad1c25b744f43b92b2316066ef044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80aefed4f78f4d2fabf8925aa34f19dd"
I20260810 14:38:09.052075 29762 leader_election.cc:302] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 523ad1c25b744f43b92b2316066ef044; no voters: 80aefed4f78f4d2fabf8925aa34f19dd, 890d7bab8502455abe2e3d2f341be9fc
I20260810 14:38:09.052352 29993 raft_consensus.cc:2726] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 14:38:09.053633 29433 raft_consensus.cc:2454] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 890d7bab8502455abe2e3d2f341be9fc in term 1.
I20260810 14:38:09.054154 29545 leader_election.cc:302] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 80aefed4f78f4d2fabf8925aa34f19dd, 890d7bab8502455abe2e3d2f341be9fc; no voters: 523ad1c25b744f43b92b2316066ef044
I20260810 14:38:09.054633 29995 raft_consensus.cc:2781] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:09.055178 29995 raft_consensus.cc:686] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 1 LEADER]: Becoming Leader. State: Replica: 890d7bab8502455abe2e3d2f341be9fc, State: Running, Role: LEADER
I20260810 14:38:09.055729 29995 consensus_queue.cc:237] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } }
I20260810 14:38:09.060674 29195 catalog_manager.cc:5495] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 890d7bab8502455abe2e3d2f341be9fc (127.21.55.66). New cstate: current_term: 1 leader_uuid: "890d7bab8502455abe2e3d2f341be9fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "890d7bab8502455abe2e3d2f341be9fc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 } health_report { overall_health: UNKNOWN } } }
W20260810 14:38:09.067688 29513 tablet.cc:2369] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:38:09.082150 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.079373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41760 (local address 127.21.55.66:38743)
0810 14:38:09.079614 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:09.079622 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:09.079680 (+ 58us) server_negotiation.cc:407] Connection header received
0810 14:38:09.079941 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:09.079949 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:09.080092 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:09.080224 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:09.081315 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:09.081323 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:09.081329 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:09.081353 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:09.081398 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:09.081415 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:09.081570 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:09.081694 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 14:38:09.081764 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260810 14:38:09.089738 29729 tablet.cc:2369] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:38:09.091943 29433 raft_consensus.cc:1261] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd [term 1 FOLLOWER]: Refusing update from remote peer 890d7bab8502455abe2e3d2f341be9fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:09.091980 29867 raft_consensus.cc:1261] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 1 FOLLOWER]: Refusing update from remote peer 890d7bab8502455abe2e3d2f341be9fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:09.092839 29995 consensus_queue.cc:1035] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "80aefed4f78f4d2fabf8925aa34f19dd" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 43615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:09.093317 30002 consensus_queue.cc:1035] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "523ad1c25b744f43b92b2316066ef044" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:09.107024 30007 mvcc.cc:205] Tried to move back new op lower bound from 7316982534508494848 to 7316982534376792064. Current Snapshot: MvccSnapshot[applied={T|T < 7316982534508494848}]
I20260810 14:38:09.111029 30010 mvcc.cc:205] Tried to move back new op lower bound from 7316982534508494848 to 7316982534376792064. Current Snapshot: MvccSnapshot[applied={T|T < 7316982534508494848}]
I20260810 14:38:09.121690 29195 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:09.121986 29195 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:09.126045 29195 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14deb85ea59946b7982dd8ada9b67ac6: Sending DeleteTablet for 3 replicas of tablet 467f127ab67943d9a7c3f4a436c5740c
I20260810 14:38:09.127170 29835 tablet_service.cc:1514] Processing DeleteTablet for tablet 467f127ab67943d9a7c3f4a436c5740c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:09 UTC) from {username='slave'} at 127.0.0.1:47166
I20260810 14:38:09.127269 29400 tablet_service.cc:1514] Processing DeleteTablet for tablet 467f127ab67943d9a7c3f4a436c5740c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:09 UTC) from {username='slave'} at 127.0.0.1:39870
I20260810 14:38:09.127278 29616 tablet_service.cc:1514] Processing DeleteTablet for tablet 467f127ab67943d9a7c3f4a436c5740c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:09 UTC) from {username='slave'} at 127.0.0.1:39850
I20260810 14:38:09.132226 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 29288
I20260810 14:38:09.138418 30025 tablet_replica.cc:330] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: stopping tablet replica
I20260810 14:38:09.139137 30025 raft_consensus.cc:2229] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:38:09.140074 30025 raft_consensus.cc:2258] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:09.141039 30022 tablet_replica.cc:330] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044: stopping tablet replica
I20260810 14:38:09.141108 30026 tablet_replica.cc:330] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd: stopping tablet replica
I20260810 14:38:09.141515 30026 raft_consensus.cc:2229] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:09.142020 30026 raft_consensus.cc:2258] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:09.142949 30022 raft_consensus.cc:2229] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:09.143363 30025 ts_tablet_manager.cc:1905] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:09.143404 30022 raft_consensus.cc:2258] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:09.144098 30026 ts_tablet_manager.cc:1905] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:09.151129 30022 ts_tablet_manager.cc:1905] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:09.151713 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.133483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45286 (local address 127.21.55.65:43615)
0810 14:38:09.133661 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:09.133668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:09.148496 (+ 14828us) server_negotiation.cc:407] Connection header received
0810 14:38:09.148919 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:09.148926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:09.149038 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:09.149120 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:09.150783 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:09.150790 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:09.150794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:09.150813 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:09.150849 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:09.150865 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:09.150962 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:09.151361 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 14:38:09.151428 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:38:09.152797 29504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:09.155776 30025 ts_tablet_manager.cc:1918] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:09.155985 30025 log.cc:1196] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/467f127ab67943d9a7c3f4a436c5740c
I20260810 14:38:09.156637 30025 ts_tablet_manager.cc:1939] T 467f127ab67943d9a7c3f4a436c5740c P 890d7bab8502455abe2e3d2f341be9fc: Deleting consensus metadata
I20260810 14:38:09.159525 29173 catalog_manager.cc:4841] TS 890d7bab8502455abe2e3d2f341be9fc (127.21.55.66:38743): tablet 467f127ab67943d9a7c3f4a436c5740c (table test-table [id=23982ed195024a30844689f4b5af942f]) successfully deleted
I20260810 14:38:09.160758 30022 ts_tablet_manager.cc:1918] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:09.161129 30022 log.cc:1196] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/467f127ab67943d9a7c3f4a436c5740c
I20260810 14:38:09.161847 30022 ts_tablet_manager.cc:1939] T 467f127ab67943d9a7c3f4a436c5740c P 523ad1c25b744f43b92b2316066ef044: Deleting consensus metadata
I20260810 14:38:09.163514 29173 catalog_manager.cc:4841] TS 523ad1c25b744f43b92b2316066ef044 (127.21.55.67:34283): tablet 467f127ab67943d9a7c3f4a436c5740c (table test-table [id=23982ed195024a30844689f4b5af942f]) successfully deleted
I20260810 14:38:09.315366 30026 ts_tablet_manager.cc:1918] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:09.315832 30026 log.cc:1196] T 467f127ab67943d9a7c3f4a436c5740c P 80aefed4f78f4d2fabf8925aa34f19dd: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/467f127ab67943d9a7c3f4a436c5740c
I20260810 14:38:09.316123 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 29288
W20260810 14:38:09.340194 29171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.55.65:43615 (error 108)
I20260810 14:38:09.341307 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 29518
I20260810 14:38:09.346206 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.342225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.21.55.66:38743)
0810 14:38:09.342416 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:09.342423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:09.342992 (+ 569us) server_negotiation.cc:407] Connection header received
0810 14:38:09.343088 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:09.343096 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:09.343184 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:09.343299 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:09.344765 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:09.344774 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:09.344781 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:09.344807 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:09.344862 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:09.344881 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:09.345037 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:09.345691 (+ 654us) server_negotiation.cc:299] Negotiation successful
0810 14:38:09.345769 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 14:38:09.347061 29719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:09.349344 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.348597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:43615 (local address 127.0.0.1:45298)
0810 14:38:09.348822 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.349125 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:43615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260810 14:38:09.349562 29171 catalog_manager.cc:4584] TS 80aefed4f78f4d2fabf8925aa34f19dd (127.21.55.65:43615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 467f127ab67943d9a7c3f4a436c5740c: Network error: Client connection negotiation failed: client connection to 127.21.55.65:43615: connect: Connection refused (error 111)
I20260810 14:38:09.403752 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.403175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:43615 (local address 127.0.0.1:45310)
0810 14:38:09.403432 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.403547 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:43615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260810 14:38:09.404872 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.404489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:43615 (local address 127.0.0.1:45314)
0810 14:38:09.404650 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.404738 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:43615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 14:38:09.465472 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.464949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:43615 (local address 127.0.0.1:45320)
0810 14:38:09.465149 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.465261 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:43615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 14:38:09.466509 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 29518
I20260810 14:38:09.466920 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.466266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:43615 (local address 127.0.0.1:45322)
0810 14:38:09.466694 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.466777 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:43615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323}
I20260810 14:38:09.494969 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 29734
I20260810 14:38:09.498373 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.495833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.21.55.67:34283)
0810 14:38:09.496040 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:09.496049 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:09.496145 (+ 96us) server_negotiation.cc:407] Connection header received
0810 14:38:09.496478 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:09.496483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:09.496569 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:09.496715 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:09.497692 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:09.497696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:09.497700 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:09.497715 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:09.497748 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:09.497761 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:09.497912 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:09.497998 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 14:38:09.498068 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 14:38:09.499346 29941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:09.548097 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.547358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:43615 (local address 127.0.0.1:45324)
0810 14:38:09.547768 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.547898 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:43615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260810 14:38:09.549333 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.548894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:43615 (local address 127.0.0.1:45328)
0810 14:38:09.549137 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.549206 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:43615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260810 14:38:09.607350 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 29734
I20260810 14:38:09.633137 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 29146
I20260810 14:38:09.634809 29244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:09.745915 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:09.745254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:43615 (local address 127.0.0.1:45340)
0810 14:38:09.745550 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:09.745680 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:43615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 14:38:09.745945 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 29146
2026-08-10T14:38:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3040 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 14:38:09.795725 21725 mini_oidc.cc:152] Starting JWKS server
I20260810 14:38:09.798784 21725 webserver.cc:460] Webserver started at https://127.0.0.1:38717/ using document root <none> and password file <none>
I20260810 14:38:09.799132 21725 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 14:38:09.799842 21725 webserver.cc:460] Webserver started at http://127.0.0.1:45519/ using document root <none> and password file <none>
2026-08-10T14:38:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:38:09Z Disabled control of system clock
I20260810 14:38:09.837606 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:35567
--webserver_interface=127.21.55.126
--webserver_port=0
--builtin_ntp_servers=127.21.55.84:38697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:35567
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38717/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/testchainca.cert with env {}
W20260810 14:38:10.205615 30047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:10.206025 30047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:10.206211 30047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:10.214404 30047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:10.214614 30047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:10.214725 30047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:10.214834 30047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:10.214913 30047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:38:10.215003 30047 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:38:10.215090 30047 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38717/jwks.json
I20260810 14:38:10.225370 30047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:38697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:35567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:35567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38717/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:10.227545 30047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:10.229979 30047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:10.246935 30058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:10.246979 30056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:10.247009 30052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:10.248512 30047 server_base.cc:1034] running on GCE node
I20260810 14:38:10.249379 30047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:10.251130 30047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:10.252696 30047 hybrid_clock.cc:648] HybridClock initialized: now 1786372690252640 us; error 58 us; skew 500 ppm
I20260810 14:38:10.253156 30047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:38:10.258773 30047 webserver.cc:460] Webserver started at http://127.21.55.126:46647/ using document root <none> and password file <none>
I20260810 14:38:10.259531 30047 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:10.259684 30047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:10.260027 30047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:10.263253 30047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "56fee3603bff4baaa8ebe5f2c5d6d0e5"
format_stamp: "Formatted at 2026-08-10 14:38:10 on dist-test-slave-h6n0"
server_key: "3f815df89b9b33f48b8fa9d129c73c7e"
server_key_iv: "e602e219f97f16172cf1934e5e5f252b"
server_key_version: "encryptionkey@0"
I20260810 14:38:10.264071 30047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "56fee3603bff4baaa8ebe5f2c5d6d0e5"
format_stamp: "Formatted at 2026-08-10 14:38:10 on dist-test-slave-h6n0"
server_key: "3f815df89b9b33f48b8fa9d129c73c7e"
server_key_iv: "e602e219f97f16172cf1934e5e5f252b"
server_key_version: "encryptionkey@0"
I20260810 14:38:10.271474 30047 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 14:38:10.276093 30070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:10.278354 30047 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260810 14:38:10.278600 30047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "56fee3603bff4baaa8ebe5f2c5d6d0e5"
format_stamp: "Formatted at 2026-08-10 14:38:10 on dist-test-slave-h6n0"
server_key: "3f815df89b9b33f48b8fa9d129c73c7e"
server_key_iv: "e602e219f97f16172cf1934e5e5f252b"
server_key_version: "encryptionkey@0"
I20260810 14:38:10.279008 30047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:10.298897 30047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:10.299923 30047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:10.300275 30047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:10.321012 30047 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:35567
I20260810 14:38:10.321035 30161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:35567 every 8 connection(s)
I20260810 14:38:10.322392 30047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:38:10.326144 30163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:10.327621 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 30047
I20260810 14:38:10.327865 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:38:10.328732 21725 external_mini_cluster.cc:1428] Setting key 15ab77d2b1b119dea1a583fb03ed1654
I20260810 14:38:10.341064 30163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5: Bootstrap starting.
I20260810 14:38:10.345043 30163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:10.346333 30163 log.cc:824] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:10.348222 30168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:10.331371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.21.55.126:35567)
0810 14:38:10.332723 (+ 1352us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:10.332734 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:10.332785 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:38:10.334764 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:10.334787 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:10.335572 (+ 785us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:10.336161 (+ 589us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:10.336942 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:10.339199 (+ 2257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:10.341413 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:10.342311 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:10.342520 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:10.344667 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:10.344678 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:10.344706 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:10.345674 (+ 968us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:10.345821 (+ 147us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:10.345854 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:10.345991 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:10.346337 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 14:38:10.346806 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":899,"thread_start_us":446,"threads_started":1}
I20260810 14:38:10.350363 30163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5: No bootstrap required, opened a new log
I20260810 14:38:10.352625 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:38:10.359898 30163 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fee3603bff4baaa8ebe5f2c5d6d0e5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35567 } }
I20260810 14:38:10.360307 30163 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:10.360416 30163 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56fee3603bff4baaa8ebe5f2c5d6d0e5, State: Initialized, Role: FOLLOWER
I20260810 14:38:10.360965 30163 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fee3603bff4baaa8ebe5f2c5d6d0e5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35567 } }
I20260810 14:38:10.361222 30163 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:38:10.361367 30163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:38:10.361536 30163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:10.364480 30163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fee3603bff4baaa8ebe5f2c5d6d0e5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35567 } }
I20260810 14:38:10.365008 30163 leader_election.cc:302] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56fee3603bff4baaa8ebe5f2c5d6d0e5; no voters:
I20260810 14:38:10.365592 30163 leader_election.cc:288] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:38:10.365811 30175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:10.367198 30175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [term 1 LEADER]: Becoming Leader. State: Replica: 56fee3603bff4baaa8ebe5f2c5d6d0e5, State: Running, Role: LEADER
I20260810 14:38:10.367866 30175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fee3603bff4baaa8ebe5f2c5d6d0e5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35567 } }
I20260810 14:38:10.368644 30163 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:10.370738 30178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56fee3603bff4baaa8ebe5f2c5d6d0e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fee3603bff4baaa8ebe5f2c5d6d0e5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35567 } } }
I20260810 14:38:10.371233 30178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:10.371115 30179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56fee3603bff4baaa8ebe5f2c5d6d0e5. Latest consensus state: current_term: 1 leader_uuid: "56fee3603bff4baaa8ebe5f2c5d6d0e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56fee3603bff4baaa8ebe5f2c5d6d0e5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35567 } } }
I20260810 14:38:10.371526 30179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:10.374161 30185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:10.378906 30185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:10.390275 30185 catalog_manager.cc:1347] Generated new cluster ID: f0b3e98eedfa4563881e6eaf0d4d9b10
I20260810 14:38:10.390426 30185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:10.401501 30185 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:38:10.403100 30185 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:10.416913 30185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56fee3603bff4baaa8ebe5f2c5d6d0e5: Generated new TSK 0
I20260810 14:38:10.417797 30185 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 14:38:10.470695 30211 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 14:38:10.470973 30211 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 14:38:10.473243 30168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:10.440401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34842 (local address 127.21.55.126:35567)
0810 14:38:10.440694 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:10.440702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:10.441097 (+ 395us) server_negotiation.cc:407] Connection header received
0810 14:38:10.441631 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:10.441636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:10.441730 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:10.441912 (+ 182us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:38:10.442590 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:10.443416 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:10.444974 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:10.445473 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:10.445627 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:10.446336 (+ 709us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:38:10.472552 (+ 26216us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38717/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 14:38:10.472951 (+ 399us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34842: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38717/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":108}
W20260810 14:38:10.473402 30168 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34842: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38717/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 14:38:10.476336 30210 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:10.440278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:35567 (local address 127.0.0.1:34842)
0810 14:38:10.440966 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:10.440996 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:10.441219 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:10.441943 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:10.441948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:10.441964 (+ 16us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:38:10.442366 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:10.442376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:10.443581 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:10.443589 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:10.444810 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:10.444824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:10.445691 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:10.445702 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:10.445857 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:10.445920 (+ 63us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:38:10.473074 (+ 27154us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38717/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 14:38:10.473733 (+ 659us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.55.126:35567: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38717/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":441,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.55.126:35567: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38717/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 14:38:10.478678 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 30047
I20260810 14:38:10.480132 30154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:10.581213 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 30047
2026-08-10T14:38:10Z chronyd exiting
I20260810 14:38:10.641806 21725 test_util.cc:175] -----------------------------------------------
I20260810 14:38:10.642005 21725 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786372641103011-21725-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (850 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 14:38:10.645761 21725 mini_oidc.cc:152] Starting JWKS server
I20260810 14:38:10.648204 21725 webserver.cc:460] Webserver started at https://127.0.0.1:39099/ using document root <none> and password file <none>
I20260810 14:38:10.648471 21725 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 14:38:10.649257 21725 webserver.cc:460] Webserver started at http://127.0.0.1:37501/ using document root <none> and password file <none>
2026-08-10T14:38:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:38:10Z Disabled control of system clock
I20260810 14:38:10.683300 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:41941
--webserver_interface=127.21.55.126
--webserver_port=0
--builtin_ntp_servers=127.21.55.84:39183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:41941
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39099/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/testchainca.cert with env {}
W20260810 14:38:11.025885 30227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:11.026261 30227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:11.026440 30227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:11.034082 30227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:11.034256 30227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:11.034354 30227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:11.034468 30227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:11.034587 30227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:38:11.034703 30227 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:38:11.034785 30227 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39099/jwks.json
I20260810 14:38:11.045081 30227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:41941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:41941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39099/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:11.046479 30227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:11.047876 30227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:11.062505 30234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:11.062664 30235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:11.062618 30239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:11.064095 30227 server_base.cc:1034] running on GCE node
I20260810 14:38:11.065393 30227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:11.067332 30227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:11.068625 30227 hybrid_clock.cc:648] HybridClock initialized: now 1786372691068567 us; error 62 us; skew 500 ppm
I20260810 14:38:11.069120 30227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:38:11.071980 30227 webserver.cc:460] Webserver started at http://127.21.55.126:42001/ using document root <none> and password file <none>
I20260810 14:38:11.072749 30227 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:11.072891 30227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:11.073244 30227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:11.076377 30227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "bc3f5dfb2da24f1ebe65f4928a4db1da"
format_stamp: "Formatted at 2026-08-10 14:38:11 on dist-test-slave-h6n0"
server_key: "abdbe707cfcd4f2aceeb2541bdcebd42"
server_key_iv: "354a4d162ede9fd58c90b32d7202742d"
server_key_version: "encryptionkey@0"
I20260810 14:38:11.077577 30227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "bc3f5dfb2da24f1ebe65f4928a4db1da"
format_stamp: "Formatted at 2026-08-10 14:38:11 on dist-test-slave-h6n0"
server_key: "abdbe707cfcd4f2aceeb2541bdcebd42"
server_key_iv: "354a4d162ede9fd58c90b32d7202742d"
server_key_version: "encryptionkey@0"
I20260810 14:38:11.085032 30227 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260810 14:38:11.089558 30246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:11.091593 30227 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 14:38:11.091877 30227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "bc3f5dfb2da24f1ebe65f4928a4db1da"
format_stamp: "Formatted at 2026-08-10 14:38:11 on dist-test-slave-h6n0"
server_key: "abdbe707cfcd4f2aceeb2541bdcebd42"
server_key_iv: "354a4d162ede9fd58c90b32d7202742d"
server_key_version: "encryptionkey@0"
I20260810 14:38:11.092147 30227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:11.105877 30227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:11.106837 30227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:11.107275 30227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:11.130179 30227 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:41941
I20260810 14:38:11.130199 30339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:41941 every 8 connection(s)
I20260810 14:38:11.131693 30227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:38:11.135895 30340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:11.137475 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 30227
I20260810 14:38:11.137701 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:38:11.138684 21725 external_mini_cluster.cc:1428] Setting key 81f1cd2de5e76500e4c10f6b97e49768
I20260810 14:38:11.152361 30340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da: Bootstrap starting.
I20260810 14:38:11.152594 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:11.140839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.21.55.126:41941)
0810 14:38:11.142071 (+ 1232us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:11.142082 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:11.142127 (+ 45us) server_negotiation.cc:407] Connection header received
0810 14:38:11.143713 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:11.143737 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:11.144218 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:11.144637 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:11.145226 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:11.146504 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:11.147834 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:11.148587 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:11.148869 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:11.150130 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:11.150138 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:11.150151 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:11.150826 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:11.150908 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:11.150936 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:11.151070 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:11.151289 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 14:38:11.151597 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":426,"threads_started":1}
I20260810 14:38:11.155994 30340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:11.156400 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:38:11.157130 30340 log.cc:824] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:11.160339 30340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da: No bootstrap required, opened a new log
I20260810 14:38:11.168749 30340 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc3f5dfb2da24f1ebe65f4928a4db1da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41941 } }
I20260810 14:38:11.169165 30340 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:11.169297 30340 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc3f5dfb2da24f1ebe65f4928a4db1da, State: Initialized, Role: FOLLOWER
I20260810 14:38:11.169914 30340 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc3f5dfb2da24f1ebe65f4928a4db1da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41941 } }
I20260810 14:38:11.170156 30340 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:38:11.170306 30340 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:38:11.170495 30340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:11.172976 30340 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc3f5dfb2da24f1ebe65f4928a4db1da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41941 } }
I20260810 14:38:11.173488 30340 leader_election.cc:302] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc3f5dfb2da24f1ebe65f4928a4db1da; no voters:
I20260810 14:38:11.174099 30340 leader_election.cc:288] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:38:11.174247 30352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:11.175043 30352 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [term 1 LEADER]: Becoming Leader. State: Replica: bc3f5dfb2da24f1ebe65f4928a4db1da, State: Running, Role: LEADER
I20260810 14:38:11.175642 30352 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc3f5dfb2da24f1ebe65f4928a4db1da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41941 } }
I20260810 14:38:11.176314 30340 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:11.178589 30353 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc3f5dfb2da24f1ebe65f4928a4db1da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc3f5dfb2da24f1ebe65f4928a4db1da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41941 } } }
I20260810 14:38:11.178560 30354 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc3f5dfb2da24f1ebe65f4928a4db1da. Latest consensus state: current_term: 1 leader_uuid: "bc3f5dfb2da24f1ebe65f4928a4db1da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc3f5dfb2da24f1ebe65f4928a4db1da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 41941 } } }
I20260810 14:38:11.179009 30353 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:11.179009 30354 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:11.179972 30362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:11.188742 30362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:11.198356 30362 catalog_manager.cc:1347] Generated new cluster ID: 971466aa00324472a73821ca9227f548
I20260810 14:38:11.198511 30362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:11.208483 30362 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:38:11.209740 30362 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:11.218559 30362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc3f5dfb2da24f1ebe65f4928a4db1da: Generated new TSK 0
I20260810 14:38:11.219369 30362 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 14:38:11.255415 30399 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 14:38:11.255554 30399 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 14:38:11.256850 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:11.235351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43106 (local address 127.21.55.126:41941)
0810 14:38:11.235559 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:11.235566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:11.236009 (+ 443us) server_negotiation.cc:407] Connection header received
0810 14:38:11.236222 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:11.236229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:11.236326 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:11.236500 (+ 174us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:38:11.236950 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:11.238087 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:11.239882 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:11.240553 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:11.240687 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:11.241210 (+ 523us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:38:11.256052 (+ 14842us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39099/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 14:38:11.256511 (+ 459us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43106: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39099/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 14:38:11.256995 30347 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43106: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39099/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 14:38:11.257109 30398 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:11.235213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:41941 (local address 127.0.0.1:43106)
0810 14:38:11.235892 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:11.235913 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:11.236041 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:11.236536 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:11.236539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:11.236549 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:38:11.236803 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:11.236811 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:11.238439 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:11.238446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:11.239731 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:11.239744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:11.240747 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:11.240751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:11.240893 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:11.240926 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:38:11.256310 (+ 15384us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39099/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 14:38:11.256681 (+ 371us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.55.126:41941: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39099/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":389,"threads_started":1}
I20260810 14:38:11.258447 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 30227
I20260810 14:38:11.259655 30333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:11.358518 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 30227
2026-08-10T14:38:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (778 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 14:38:11.425004 21725 mini_oidc.cc:152] Starting JWKS server
I20260810 14:38:11.427106 21725 webserver.cc:460] Webserver started at https://127.0.0.1:34627/ using document root <none> and password file <none>
I20260810 14:38:11.427379 21725 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 14:38:11.428113 21725 webserver.cc:460] Webserver started at http://127.0.0.1:45759/ using document root <none> and password file <none>
2026-08-10T14:38:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:38:11Z Disabled control of system clock
I20260810 14:38:11.467511 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:39307
--webserver_interface=127.21.55.126
--webserver_port=0
--builtin_ntp_servers=127.21.55.84:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:39307
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34627/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 14:38:11.837656 30407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:11.838171 30407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:11.838382 30407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:11.848023 30407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:11.848265 30407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:11.848371 30407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:11.848495 30407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:11.848587 30407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:38:11.848682 30407 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:38:11.848762 30407 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34627/jwks.json
I20260810 14:38:11.859331 30407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:39307
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:39307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34627/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:11.860736 30407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:11.862582 30407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:11.875101 30413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:11.875075 30418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:11.875048 30422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:11.880688 30407 server_base.cc:1034] running on GCE node
I20260810 14:38:11.881642 30407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:11.883347 30407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:11.884694 30407 hybrid_clock.cc:648] HybridClock initialized: now 1786372691884605 us; error 73 us; skew 500 ppm
I20260810 14:38:11.885264 30407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:38:11.888790 30407 webserver.cc:460] Webserver started at http://127.21.55.126:44981/ using document root <none> and password file <none>
I20260810 14:38:11.889775 30407 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:11.889940 30407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:11.890671 30407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:11.894810 30407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "7b1b087496574b65b026eb83e847f570"
format_stamp: "Formatted at 2026-08-10 14:38:11 on dist-test-slave-h6n0"
server_key: "32fa06b34aab3e199c5c65f78512d0d5"
server_key_iv: "e63efd27c4c9088ca8b1f410960b2aae"
server_key_version: "encryptionkey@0"
I20260810 14:38:11.895675 30407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "7b1b087496574b65b026eb83e847f570"
format_stamp: "Formatted at 2026-08-10 14:38:11 on dist-test-slave-h6n0"
server_key: "32fa06b34aab3e199c5c65f78512d0d5"
server_key_iv: "e63efd27c4c9088ca8b1f410960b2aae"
server_key_version: "encryptionkey@0"
I20260810 14:38:11.901985 30407 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260810 14:38:11.906639 30431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:11.908838 30407 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260810 14:38:11.909142 30407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "7b1b087496574b65b026eb83e847f570"
format_stamp: "Formatted at 2026-08-10 14:38:11 on dist-test-slave-h6n0"
server_key: "32fa06b34aab3e199c5c65f78512d0d5"
server_key_iv: "e63efd27c4c9088ca8b1f410960b2aae"
server_key_version: "encryptionkey@0"
I20260810 14:38:11.909593 30407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:11.923837 30407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:11.924690 30407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:11.925035 30407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:11.945708 30407 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:39307
I20260810 14:38:11.945747 30511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:39307 every 8 connection(s)
I20260810 14:38:11.947114 30407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:38:11.951021 30513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:11.954906 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 30407
I20260810 14:38:11.955134 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:38:11.955968 21725 external_mini_cluster.cc:1428] Setting key 18d02c9960811433b6764fddaf38faff
I20260810 14:38:11.966024 30513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570: Bootstrap starting.
I20260810 14:38:11.970568 30513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:11.971460 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:11.957892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59496 (local address 127.21.55.126:39307)
0810 14:38:11.958971 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:11.958982 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:11.959022 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:38:11.960609 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:11.960629 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:11.961091 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:11.961413 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:11.962674 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:11.964333 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:11.966611 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:11.967310 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:11.967470 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:11.969061 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:11.969075 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:11.969087 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:11.969616 (+ 529us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:11.969709 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:11.969730 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:11.969813 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:11.970079 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 14:38:11.970408 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":408,"threads_started":1}
I20260810 14:38:11.971702 30513 log.cc:824] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:11.975117 30513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570: No bootstrap required, opened a new log
I20260810 14:38:11.975865 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:38:11.983989 30513 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1b087496574b65b026eb83e847f570" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39307 } }
I20260810 14:38:11.984385 30513 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:11.984503 30513 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b1b087496574b65b026eb83e847f570, State: Initialized, Role: FOLLOWER
I20260810 14:38:11.985286 30513 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1b087496574b65b026eb83e847f570" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39307 } }
I20260810 14:38:11.985566 30513 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:38:11.985721 30513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:38:11.985903 30513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:11.988209 30513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1b087496574b65b026eb83e847f570" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39307 } }
I20260810 14:38:11.988690 30513 leader_election.cc:302] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b1b087496574b65b026eb83e847f570; no voters:
I20260810 14:38:11.989269 30513 leader_election.cc:288] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:38:11.989410 30530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:11.990197 30530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [term 1 LEADER]: Becoming Leader. State: Replica: 7b1b087496574b65b026eb83e847f570, State: Running, Role: LEADER
I20260810 14:38:11.990799 30530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1b087496574b65b026eb83e847f570" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39307 } }
I20260810 14:38:11.991457 30513 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:11.993794 30533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b1b087496574b65b026eb83e847f570. Latest consensus state: current_term: 1 leader_uuid: "7b1b087496574b65b026eb83e847f570" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1b087496574b65b026eb83e847f570" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39307 } } }
I20260810 14:38:11.994436 30533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:11.993928 30531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b1b087496574b65b026eb83e847f570" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1b087496574b65b026eb83e847f570" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39307 } } }
I20260810 14:38:11.994731 30531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:11.999712 30543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:12.003819 30543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:12.011200 30543 catalog_manager.cc:1347] Generated new cluster ID: a2e5d875e3064bd48c6cdf85ff9edd9e
I20260810 14:38:12.011309 30543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:12.035452 30543 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:38:12.036803 30543 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:12.046633 30543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b1b087496574b65b026eb83e847f570: Generated new TSK 0
I20260810 14:38:12.047487 30543 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:38:12.079727 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:12.070483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59506 (local address 127.21.55.126:39307)
0810 14:38:12.070767 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:12.070777 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:12.071281 (+ 504us) server_negotiation.cc:407] Connection header received
0810 14:38:12.071489 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:12.071493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:12.071599 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:12.071774 (+ 175us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:38:12.072446 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:12.073266 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:12.075373 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:12.075872 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:12.075976 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:12.076594 (+ 618us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:38:12.079045 (+ 2451us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34627/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 14:38:12.079394 (+ 349us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59506: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34627/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":86}
W20260810 14:38:12.079861 30523 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59506: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34627/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 14:38:12.081852 30572 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:12.070437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:39307 (local address 127.0.0.1:59506)
0810 14:38:12.071094 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:12.071115 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:12.071269 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:12.071822 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:12.071830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:12.071844 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:38:12.072225 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:12.072237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:12.073421 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:12.073427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:12.075216 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:12.075230 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:12.076079 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:12.076087 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:12.076267 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:12.076313 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:38:12.079273 (+ 2960us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34627/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 14:38:12.081400 (+ 2127us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.55.126:39307: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34627/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":1783,"thread_start_us":373,"threads_started":1}
I20260810 14:38:12.083350 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 30407
I20260810 14:38:12.085219 30505 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 14:38:12.092664 30574 webserver.cc:533] Webserver: sslize failed:
E20260810 14:38:12.092795 30574 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 14:38:12.180671 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 30407
2026-08-10T14:38:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (817 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T14:38:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:38:12Z Disabled control of system clock
I20260810 14:38:12.278299 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:35835
--webserver_interface=127.21.55.126
--webserver_port=0
--builtin_ntp_servers=127.21.55.84:36141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:35835
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/testchainca.cert with env {}
W20260810 14:38:12.620007 30580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:12.620357 30580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:12.620528 30580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:12.627683 30580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:12.627825 30580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:12.627920 30580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:12.628027 30580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:12.628149 30580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:38:12.628245 30580 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:38:12.628304 30580 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 14:38:12.639190 30580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:36141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:35835
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:35835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:12.640684 30580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:12.642041 30580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:12.655038 30588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:12.654951 30591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:12.655014 30589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:12.655767 30580 server_base.cc:1034] running on GCE node
I20260810 14:38:12.656646 30580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:12.658330 30580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:12.659750 30580 hybrid_clock.cc:648] HybridClock initialized: now 1786372692659655 us; error 84 us; skew 500 ppm
I20260810 14:38:12.660418 30580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:38:12.663854 30580 webserver.cc:460] Webserver started at http://127.21.55.126:39343/ using document root <none> and password file <none>
I20260810 14:38:12.664638 30580 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:12.664788 30580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:12.665402 30580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:12.668653 30580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "82c4144d3e1440aeaa7d02662eacf333"
format_stamp: "Formatted at 2026-08-10 14:38:12 on dist-test-slave-h6n0"
server_key: "a6deaa9103076cda7078e09721b2a5f9"
server_key_iv: "ec2bb0e476e6421dfbe3555964a9dc4c"
server_key_version: "encryptionkey@0"
I20260810 14:38:12.669509 30580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "82c4144d3e1440aeaa7d02662eacf333"
format_stamp: "Formatted at 2026-08-10 14:38:12 on dist-test-slave-h6n0"
server_key: "a6deaa9103076cda7078e09721b2a5f9"
server_key_iv: "ec2bb0e476e6421dfbe3555964a9dc4c"
server_key_version: "encryptionkey@0"
I20260810 14:38:12.674758 30580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260810 14:38:12.678236 30602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:12.680552 30580 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:38:12.680915 30580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "82c4144d3e1440aeaa7d02662eacf333"
format_stamp: "Formatted at 2026-08-10 14:38:12 on dist-test-slave-h6n0"
server_key: "a6deaa9103076cda7078e09721b2a5f9"
server_key_iv: "ec2bb0e476e6421dfbe3555964a9dc4c"
server_key_version: "encryptionkey@0"
I20260810 14:38:12.681358 30580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:12.701481 30580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:12.702713 30580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:12.703047 30580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:12.723856 30580 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:35835
I20260810 14:38:12.723875 30684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:35835 every 8 connection(s)
I20260810 14:38:12.725270 30580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:38:12.729044 30686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:12.733771 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 30580
I20260810 14:38:12.734000 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:38:12.734879 21725 external_mini_cluster.cc:1428] Setting key 8cf480bb292d46f05a52cabd0b988fd3
I20260810 14:38:12.741118 30686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333: Bootstrap starting.
I20260810 14:38:12.744737 30686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:12.745909 30686 log.cc:824] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:12.750173 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:12.737290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59022 (local address 127.21.55.126:35835)
0810 14:38:12.738348 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:12.738362 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:12.738408 (+ 46us) server_negotiation.cc:407] Connection header received
0810 14:38:12.740149 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:12.740171 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:12.740604 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:12.740917 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:12.741559 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:12.742991 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:12.745039 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:12.745739 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:12.745869 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:12.747734 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:12.747741 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:12.747754 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:38:12.748316 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:12.748393 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:38:12.748415 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:12.748507 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:12.748851 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 14:38:12.749168 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":409,"threads_started":1}
I20260810 14:38:12.750286 30686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333: No bootstrap required, opened a new log
I20260810 14:38:12.754822 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:38:12.760025 30686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4144d3e1440aeaa7d02662eacf333" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35835 } }
I20260810 14:38:12.760356 30686 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:12.760435 30686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c4144d3e1440aeaa7d02662eacf333, State: Initialized, Role: FOLLOWER
I20260810 14:38:12.760888 30686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4144d3e1440aeaa7d02662eacf333" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35835 } }
I20260810 14:38:12.761080 30686 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:38:12.761173 30686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:38:12.761296 30686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:12.763646 30686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4144d3e1440aeaa7d02662eacf333" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35835 } }
I20260810 14:38:12.764163 30686 leader_election.cc:302] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82c4144d3e1440aeaa7d02662eacf333; no voters:
I20260810 14:38:12.764880 30686 leader_election.cc:288] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:38:12.765256 30701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:12.767177 30701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [term 1 LEADER]: Becoming Leader. State: Replica: 82c4144d3e1440aeaa7d02662eacf333, State: Running, Role: LEADER
I20260810 14:38:12.767474 30686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:12.768005 30701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4144d3e1440aeaa7d02662eacf333" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35835 } }
I20260810 14:38:12.771143 30704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82c4144d3e1440aeaa7d02662eacf333. Latest consensus state: current_term: 1 leader_uuid: "82c4144d3e1440aeaa7d02662eacf333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4144d3e1440aeaa7d02662eacf333" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35835 } } }
I20260810 14:38:12.771548 30704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:12.772142 30703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82c4144d3e1440aeaa7d02662eacf333" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c4144d3e1440aeaa7d02662eacf333" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35835 } } }
I20260810 14:38:12.773293 30703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:12.776055 30720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:12.781181 30720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:12.790511 30720 catalog_manager.cc:1347] Generated new cluster ID: 4f10936749574058b611806fc631c7b1
I20260810 14:38:12.790757 30720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:12.817853 30720 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:38:12.819216 30720 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:12.826912 30720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82c4144d3e1440aeaa7d02662eacf333: Generated new TSK 0
I20260810 14:38:12.827701 30720 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:38:13.352339 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:12.849754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59028 (local address 127.21.55.126:35835)
0810 14:38:12.849994 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:12.850005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:12.850626 (+ 621us) server_negotiation.cc:407] Connection header received
0810 14:38:12.850749 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:12.850753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:12.850827 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:12.850975 (+ 148us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:38:12.851513 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:12.852296 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:12.854419 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:12.855001 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:12.855108 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:12.855705 (+ 597us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:38:13.351762 (+496057us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 14:38:13.352103 (+ 341us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59028: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":77}
W20260810 14:38:13.352499 30696 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59028: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 14:38:13.353004 30749 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:12.849732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:35835 (local address 127.0.0.1:59028)
0810 14:38:12.850389 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:12.850410 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:12.850605 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:12.851012 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:12.851018 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:12.851030 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:38:12.851347 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:12.851359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:12.852440 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:12.852446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:12.854201 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:12.854217 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:12.855187 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:12.855194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:12.855395 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:12.855443 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:38:13.352073 (+496630us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 14:38:13.352344 (+ 271us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.55.126:35835: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":349,"threads_started":1}
I20260810 14:38:13.355104 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 30580
I20260810 14:38:13.356217 30677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:13.458775 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 30580
2026-08-10T14:38:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1257 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (426 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (441 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:38:14 dist-test-slave-h6n0 krb5kdc[30772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:38:14 dist-test-slave-h6n0 krb5kdc[30772](info): set up 2 sockets
Aug 10 14:38:14 dist-test-slave-h6n0 krb5kdc[30772](info): commencing operation
krb5kdc: starting...
W20260810 14:38:15.685703 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:38:15 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:38:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:38:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:38:15.936563 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:44675
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:44675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:16.288112 30794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:16.288458 30794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:16.288622 30794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:16.295734 30794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:16.295878 30794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:16.295992 30794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:16.296099 30794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:16.296192 30794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:38:16.305994 30794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:44675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:44675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:16.307582 30794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:16.309194 30794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:16.322424 30806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:16.322124 30801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:16.322907 30802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:16.323879 30794 server_base.cc:1034] running on GCE node
I20260810 14:38:16.324666 30794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:16.326056 30794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:16.327334 30794 hybrid_clock.cc:648] HybridClock initialized: now 1786372696327285 us; error 32 us; skew 500 ppm
I20260810 14:38:16.327740 30794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:16 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372696, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:16.335274 30794 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:38:16.336891 30794 webserver.cc:460] Webserver started at http://127.21.55.126:42385/ using document root <none> and password file <none>
I20260810 14:38:16.337597 30794 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:16.337752 30794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:16.338086 30794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:16.341089 30794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "993de463a0524a0e9b8589eed0d07ea5"
format_stamp: "Formatted at 2026-08-10 14:38:16 on dist-test-slave-h6n0"
server_key: "5d427971549e7896868d9527422ca33d"
server_key_iv: "93697074587c1c29731854b337c7d82d"
server_key_version: "encryptionkey@0"
I20260810 14:38:16.341863 30794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "993de463a0524a0e9b8589eed0d07ea5"
format_stamp: "Formatted at 2026-08-10 14:38:16 on dist-test-slave-h6n0"
server_key: "5d427971549e7896868d9527422ca33d"
server_key_iv: "93697074587c1c29731854b337c7d82d"
server_key_version: "encryptionkey@0"
I20260810 14:38:16.347005 30794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:38:16.350425 30818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:16.351763 30794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:38:16.351964 30794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "993de463a0524a0e9b8589eed0d07ea5"
format_stamp: "Formatted at 2026-08-10 14:38:16 on dist-test-slave-h6n0"
server_key: "5d427971549e7896868d9527422ca33d"
server_key_iv: "93697074587c1c29731854b337c7d82d"
server_key_version: "encryptionkey@0"
I20260810 14:38:16.352164 30794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:16.368029 30794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:16.372480 30794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:16.372788 30794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:16.393298 30794 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:44675
I20260810 14:38:16.393318 30910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:44675 every 8 connection(s)
I20260810 14:38:16.394733 30794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:38:16.398679 30911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:16.400831 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 30794
I20260810 14:38:16.401064 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:38:16.401881 21725 external_mini_cluster.cc:1428] Setting key 7768535b7eb452bcaca7bf0d68068917
I20260810 14:38:16.412475 30911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5: Bootstrap starting.
I20260810 14:38:16.416882 30911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:16.417976 30911 log.cc:824] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:38:16 dist-test-slave-h6n0 krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:16.423085 30911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5: No bootstrap required, opened a new log
I20260810 14:38:16.431041 30914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:16.404082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36794 (local address 127.21.55.126:44675)
0810 14:38:16.404886 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:16.404901 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:16.404942 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:38:16.406730 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:16.406755 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:16.407240 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:16.407680 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:16.410820 (+ 3140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:16.412230 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:16.413730 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:16.414517 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:16.414815 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:16.421439 (+ 6624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:16.421447 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:16.421465 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:16.421507 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:16.425712 (+ 4205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:16.427048 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:16.427057 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:16.427063 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:16.427255 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:16.427801 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:16.427805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:16.427814 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:16.428500 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:16.428743 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:16.429289 (+ 546us) server_negotiation.cc:299] Negotiation successful
0810 14:38:16.429789 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":374,"threads_started":1}
I20260810 14:38:16.433476 30911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993de463a0524a0e9b8589eed0d07ea5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44675 } }
I20260810 14:38:16.433913 30911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:16.434162 30911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 993de463a0524a0e9b8589eed0d07ea5, State: Initialized, Role: FOLLOWER
I20260810 14:38:16.434818 30911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993de463a0524a0e9b8589eed0d07ea5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44675 } }
I20260810 14:38:16.435043 30911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:38:16.435225 30911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:38:16.435391 30911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:16.437834 30911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993de463a0524a0e9b8589eed0d07ea5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44675 } }
I20260810 14:38:16.438331 30911 leader_election.cc:302] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 993de463a0524a0e9b8589eed0d07ea5; no voters:
I20260810 14:38:16.439067 30911 leader_election.cc:288] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:38:16.439250 30917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:16.440034 30917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [term 1 LEADER]: Becoming Leader. State: Replica: 993de463a0524a0e9b8589eed0d07ea5, State: Running, Role: LEADER
I20260810 14:38:16.440616 30917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993de463a0524a0e9b8589eed0d07ea5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44675 } }
I20260810 14:38:16.441092 30911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:16.443622 30919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "993de463a0524a0e9b8589eed0d07ea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993de463a0524a0e9b8589eed0d07ea5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44675 } } }
I20260810 14:38:16.444118 30919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:16.443936 30922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 993de463a0524a0e9b8589eed0d07ea5. Latest consensus state: current_term: 1 leader_uuid: "993de463a0524a0e9b8589eed0d07ea5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993de463a0524a0e9b8589eed0d07ea5" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44675 } } }
I20260810 14:38:16.444619 30922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:16.446777 30928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:16.456722 30928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:16.467550 30928 catalog_manager.cc:1347] Generated new cluster ID: 264004f157f140349741255d769408ca
I20260810 14:38:16.467689 30928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:16.483654 30928 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:38:16.485085 30928 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:16.494649 30928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 993de463a0524a0e9b8589eed0d07ea5: Generated new TSK 0
I20260810 14:38:16.495513 30928 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:38:16.560561 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:44675
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:16.920316 30950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:16.920672 30950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:16.920857 30950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:16.928262 30950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:16.928442 30950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:16.928628 30950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:38:16.938800 30950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:44675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:16.940177 30950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:16.941625 30950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:16.955399 30965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:16.955331 30962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:16.955473 30968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:16.960647 30950 server_base.cc:1034] running on GCE node
I20260810 14:38:16.961355 30950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:16.962329 30950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:16.963613 30950 hybrid_clock.cc:648] HybridClock initialized: now 1786372696963544 us; error 46 us; skew 500 ppm
I20260810 14:38:16.964083 30950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:16 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372696, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:16.967708 30950 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:38:16.969976 30950 webserver.cc:460] Webserver started at http://127.21.55.65:44449/ using document root <none> and password file <none>
I20260810 14:38:16.971067 30950 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:16.971235 30950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:16.971573 30950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:16.974509 30950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "51943b109ce840a0bdb4eb86b63d0e33"
format_stamp: "Formatted at 2026-08-10 14:38:16 on dist-test-slave-h6n0"
server_key: "ef48fc1346d9faf8f78213eca1df649d"
server_key_iv: "5006d5377042346e35a2eaf29816cdbc"
server_key_version: "encryptionkey@0"
I20260810 14:38:16.975405 30950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "51943b109ce840a0bdb4eb86b63d0e33"
format_stamp: "Formatted at 2026-08-10 14:38:16 on dist-test-slave-h6n0"
server_key: "ef48fc1346d9faf8f78213eca1df649d"
server_key_iv: "5006d5377042346e35a2eaf29816cdbc"
server_key_version: "encryptionkey@0"
I20260810 14:38:16.980545 30950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260810 14:38:16.984010 30978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:16.985769 30950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:38:16.986074 30950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "51943b109ce840a0bdb4eb86b63d0e33"
format_stamp: "Formatted at 2026-08-10 14:38:16 on dist-test-slave-h6n0"
server_key: "ef48fc1346d9faf8f78213eca1df649d"
server_key_iv: "5006d5377042346e35a2eaf29816cdbc"
server_key_version: "encryptionkey@0"
I20260810 14:38:16.986289 30950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:16.998739 30950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:17.003252 30950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:17.003561 30950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:17.004513 30950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:17.005920 30950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:17.006028 30950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:17.006176 30950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:17.006253 30950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:17.047569 30950 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:34513
I20260810 14:38:17.047657 31172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:34513 every 8 connection(s)
I20260810 14:38:17.048883 30950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:38:17.056782 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 30950
I20260810 14:38:17.057050 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:38:17.057972 21725 external_mini_cluster.cc:1428] Setting key c562d6396cf3d0d2dda839c68bf54eb7
Aug 10 14:38:17 dist-test-slave-h6n0 krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372696, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:17.073827 31175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:17.051847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:51393 (local address 127.21.55.126:44675)
0810 14:38:17.052381 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:17.052386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:17.053097 (+ 711us) server_negotiation.cc:407] Connection header received
0810 14:38:17.055205 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:17.055219 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:17.055345 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:17.055487 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:17.057728 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:17.058518 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:17.060815 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:17.061560 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:17.061716 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:17.066814 (+ 5098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:17.066824 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:17.066832 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:17.066876 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:17.070047 (+ 3171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:17.071122 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:17.071126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:17.071129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:17.071286 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:17.071837 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:17.071841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:17.071842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:17.072109 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:17.072486 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:17.073390 (+ 904us) server_negotiation.cc:299] Negotiation successful
0810 14:38:17.073562 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":293,"threads_started":1}
I20260810 14:38:17.075505 31176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:17.052167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:44675 (local address 127.21.55.65:51393)
0810 14:38:17.052922 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:17.052977 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:17.054915 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:17.055732 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:17.055740 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:17.056633 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:17.057535 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:17.057563 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:17.058751 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:17.058760 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:17.060652 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:17.060666 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:17.061806 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:17.061816 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:17.062028 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:17.063082 (+ 1054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:17.063127 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:17.066505 (+ 3378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:17.070187 (+ 3682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:17.070200 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:17.070218 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:17.070972 (+ 754us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:17.071416 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:17.071425 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:17.071432 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:17.071698 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:17.072404 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:17.072416 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:17.072916 (+ 500us) client_negotiation.cc:769] Sending connection context
0810 14:38:17.073240 (+ 324us) client_negotiation.cc:240] Negotiation successful
0810 14:38:17.073856 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":301,"threads_started":1}
I20260810 14:38:17.077553 31173 heartbeater.cc:347] Connected to a master server at 127.21.55.126:44675
I20260810 14:38:17.077907 31173 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:17.078704 31173 heartbeater.cc:511] Master 127.21.55.126:44675 requested a full tablet report, sending...
I20260810 14:38:17.081167 30845 ts_manager.cc:194] Registered new tserver with Master: 51943b109ce840a0bdb4eb86b63d0e33 (127.21.55.65:34513)
I20260810 14:38:17.083659 30845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:51393
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:38:17.118511 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:44675
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:17.479317 31182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:17.479633 31182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:17.479806 31182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:17.488219 31182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:17.488466 31182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:17.488672 31182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:38:17.499815 31182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:44675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:17.501173 31182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:17.502686 31182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:17.516845 31188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:17.516867 31199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:17.516809 31193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:17.517704 31182 server_base.cc:1034] running on GCE node
I20260810 14:38:17.518256 31182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:17.519232 31182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:17.520467 31182 hybrid_clock.cc:648] HybridClock initialized: now 1786372697520392 us; error 87 us; skew 500 ppm
I20260810 14:38:17.520737 31182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:17 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:17.523866 31182 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:38:17.525393 31182 webserver.cc:460] Webserver started at http://127.21.55.66:42391/ using document root <none> and password file <none>
I20260810 14:38:17.526006 31182 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:17.526118 31182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:17.526366 31182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:17.529273 31182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "6c46b5cba3b04fbea07018e1241a5dc0"
format_stamp: "Formatted at 2026-08-10 14:38:17 on dist-test-slave-h6n0"
server_key: "aa44e0a8f6d1f077a016890d846b79df"
server_key_iv: "68ec6690b1cc69bbf874fb2e97293925"
server_key_version: "encryptionkey@0"
I20260810 14:38:17.529986 31182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "6c46b5cba3b04fbea07018e1241a5dc0"
format_stamp: "Formatted at 2026-08-10 14:38:17 on dist-test-slave-h6n0"
server_key: "aa44e0a8f6d1f077a016890d846b79df"
server_key_iv: "68ec6690b1cc69bbf874fb2e97293925"
server_key_version: "encryptionkey@0"
I20260810 14:38:17.535027 31182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 14:38:17.538434 31212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:17.540251 31182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:38:17.540490 31182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "6c46b5cba3b04fbea07018e1241a5dc0"
format_stamp: "Formatted at 2026-08-10 14:38:17 on dist-test-slave-h6n0"
server_key: "aa44e0a8f6d1f077a016890d846b79df"
server_key_iv: "68ec6690b1cc69bbf874fb2e97293925"
server_key_version: "encryptionkey@0"
I20260810 14:38:17.540819 31182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:17.575198 31182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:17.579499 31182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:17.579823 31182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:17.580832 31182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:17.582218 31182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:17.582298 31182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:17.582430 31182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:17.582494 31182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:17.624430 31182 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:40961
I20260810 14:38:17.624679 31410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:40961 every 8 connection(s)
I20260810 14:38:17.625908 31182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:38:17.634979 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 31182
I20260810 14:38:17.635144 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:38:17.635600 21725 external_mini_cluster.cc:1428] Setting key 806eca82dcfbda5d8a3ca327ae4153f5
Aug 10 14:38:17 dist-test-slave-h6n0 krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:17.650902 31413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:17.628741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:46725 (local address 127.21.55.126:44675)
0810 14:38:17.629226 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:17.629231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:17.630076 (+ 845us) server_negotiation.cc:407] Connection header received
0810 14:38:17.632177 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:17.632189 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:17.632333 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:17.632523 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:17.634836 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:17.635911 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:17.637888 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:17.638329 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:17.638425 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:17.643013 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:17.643018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:17.643021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:17.643066 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:17.646286 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:17.647456 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:17.647463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:17.647467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:17.647633 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:17.648626 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:17.648631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:17.648633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:17.648959 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:17.649109 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:17.650358 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0810 14:38:17.650630 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":278,"threads_started":1}
I20260810 14:38:17.652458 31417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:17.629195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:44675 (local address 127.21.55.66:46725)
0810 14:38:17.629897 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:17.629948 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:17.631712 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:17.632919 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:17.632931 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:17.633864 (+ 933us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:17.634523 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:17.634553 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:17.636099 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:17.636105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:17.637709 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:17.637722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:17.638618 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:17.638628 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:17.638961 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:17.639749 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:17.639785 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:17.642778 (+ 2993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:17.646577 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:17.646592 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:17.646614 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:17.647213 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:17.647887 (+ 674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:17.647905 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:17.647914 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:17.648426 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:17.649261 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:17.649279 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:17.649782 (+ 503us) client_negotiation.cc:769] Sending connection context
0810 14:38:17.650252 (+ 470us) client_negotiation.cc:240] Negotiation successful
0810 14:38:17.650948 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":303,"threads_started":1}
I20260810 14:38:17.653992 31411 heartbeater.cc:347] Connected to a master server at 127.21.55.126:44675
I20260810 14:38:17.654296 31411 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:17.655108 31411 heartbeater.cc:511] Master 127.21.55.126:44675 requested a full tablet report, sending...
I20260810 14:38:17.656675 30845 ts_manager.cc:194] Registered new tserver with Master: 6c46b5cba3b04fbea07018e1241a5dc0 (127.21.55.66:40961)
I20260810 14:38:17.657614 30845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:46725
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:38:17.696141 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:44675
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:18.061172 31423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:18.061447 31423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:18.061594 31423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:18.069346 31423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:18.069547 31423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:18.069784 31423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:38:18.079677 31423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:44675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:18.080890 31423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:18.082114 31423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:38:18.087180 31173 heartbeater.cc:503] Master 127.21.55.126:44675 was elected leader, sending a full tablet report...
W20260810 14:38:18.097066 31438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:18.096989 31436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:18.097196 31434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:18.097829 31423 server_base.cc:1034] running on GCE node
I20260810 14:38:18.098670 31423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:18.099840 31423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:18.103229 31423 hybrid_clock.cc:648] HybridClock initialized: now 1786372698103119 us; error 113 us; skew 500 ppm
I20260810 14:38:18.103627 31423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:18.107304 31423 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:38:18.108888 31423 webserver.cc:460] Webserver started at http://127.21.55.67:34143/ using document root <none> and password file <none>
I20260810 14:38:18.109577 31423 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:18.109690 31423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:18.109961 31423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:18.113252 31423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "7aef92dda75b4068b38edf4df3dee106"
format_stamp: "Formatted at 2026-08-10 14:38:18 on dist-test-slave-h6n0"
server_key: "b58c8fe174defba69e7ed4df66fd2b41"
server_key_iv: "591a41bc4a216bca76127396e602b4d4"
server_key_version: "encryptionkey@0"
I20260810 14:38:18.114051 31423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "7aef92dda75b4068b38edf4df3dee106"
format_stamp: "Formatted at 2026-08-10 14:38:18 on dist-test-slave-h6n0"
server_key: "b58c8fe174defba69e7ed4df66fd2b41"
server_key_iv: "591a41bc4a216bca76127396e602b4d4"
server_key_version: "encryptionkey@0"
I20260810 14:38:18.119746 31423 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:38:18.123420 31451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:18.124955 31423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:38:18.125174 31423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "7aef92dda75b4068b38edf4df3dee106"
format_stamp: "Formatted at 2026-08-10 14:38:18 on dist-test-slave-h6n0"
server_key: "b58c8fe174defba69e7ed4df66fd2b41"
server_key_iv: "591a41bc4a216bca76127396e602b4d4"
server_key_version: "encryptionkey@0"
I20260810 14:38:18.125386 31423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:18.140077 31423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:18.144812 31423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:18.145136 31423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:18.146297 31423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:18.147894 31423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:18.147974 31423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:18.148128 31423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:18.148195 31423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:18.193675 31423 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:41035
I20260810 14:38:18.193800 31642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:41035 every 8 connection(s)
I20260810 14:38:18.195562 31423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:38:18.200968 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 31423
I20260810 14:38:18.201391 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:38:18.202174 21725 external_mini_cluster.cc:1428] Setting key 9fa6a5cb5ef4d18cb454fef54cd7016b
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:18.221912 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:18.198995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:54611 (local address 127.21.55.126:44675)
0810 14:38:18.199674 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:18.199680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:18.200656 (+ 976us) server_negotiation.cc:407] Connection header received
0810 14:38:18.203348 (+ 2692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:18.203356 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:18.203507 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:18.203657 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:18.206230 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.207124 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.208797 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.209201 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.209300 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:18.213607 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:18.213613 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:18.213616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:18.213651 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:18.216740 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.218022 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.218026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.218029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.218240 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.219412 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.219422 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.219428 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.219889 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:18.220100 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:18.221055 (+ 955us) server_negotiation.cc:299] Negotiation successful
0810 14:38:18.221422 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":306,"threads_started":1}
I20260810 14:38:18.223446 31650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:18.199444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:44675 (local address 127.21.55.67:54611)
0810 14:38:18.200401 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:18.200459 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:18.202802 (+ 2343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:18.204292 (+ 1490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:18.204308 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:18.205028 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:18.205970 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:18.206001 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.207315 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.207320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:18.208632 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:18.208646 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.209447 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.209454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:18.209689 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:18.210522 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:18.210623 (+ 101us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:18.213385 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:18.217022 (+ 3637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:18.217034 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:18.217054 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:18.217813 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:18.218804 (+ 991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:18.218809 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:18.218812 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:18.219099 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:18.220053 (+ 954us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:18.220063 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:18.220588 (+ 525us) client_negotiation.cc:769] Sending connection context
0810 14:38:18.220937 (+ 349us) client_negotiation.cc:240] Negotiation successful
0810 14:38:18.221652 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":353,"threads_started":1}
I20260810 14:38:18.225694 31644 heartbeater.cc:347] Connected to a master server at 127.21.55.126:44675
I20260810 14:38:18.226183 31644 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:18.227382 31644 heartbeater.cc:511] Master 127.21.55.126:44675 requested a full tablet report, sending...
I20260810 14:38:18.229630 30845 ts_manager.cc:194] Registered new tserver with Master: 7aef92dda75b4068b38edf4df3dee106 (127.21.55.67:41035)
I20260810 14:38:18.231478 30845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:54611
I20260810 14:38:18.233296 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/test-admin.keytab.
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:18.270262 21725 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 14:38:18.279126 31658 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:18.274744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55298 (local address 127.21.55.65:34513)
0810 14:38:18.275906 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:18.275924 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:18.276639 (+ 715us) server_negotiation.cc:407] Connection header received
0810 14:38:18.276816 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:18.276829 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:18.277157 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:18.277352 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:18.278558 (+ 1206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55298: BlockingRecv error: recv got EOF from 127.0.0.1:55298 (error 108)
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":436,"threads_started":1}
W20260810 14:38:18.280440 21725 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/krb5cc' of type FILE
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:18.284094 21725 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:18.287165 21725 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:38:18.311666 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:18.289678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55310 (local address 127.21.55.65:34513)
0810 14:38:18.290111 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:18.290119 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:18.290623 (+ 504us) server_negotiation.cc:407] Connection header received
0810 14:38:18.291163 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:18.291168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:18.291243 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:18.291338 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:18.292658 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.294286 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.296357 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.297713 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.298160 (+ 447us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:18.303001 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:18.303016 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:18.303035 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:18.303082 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:18.307238 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.308274 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.308285 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.308288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.308514 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.309058 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.309066 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.309069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.309528 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:18.309776 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:18.310826 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0810 14:38:18.311350 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
W20260810 14:38:18.320253 31658 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:18.317678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55312 (local address 127.21.55.65:34513)
0810 14:38:18.317914 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:18.317919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:18.318422 (+ 503us) server_negotiation.cc:407] Connection header received
0810 14:38:18.318656 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:18.318660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:18.318747 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:18.318868 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:18.319903 (+ 1035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55312: BlockingRecv error: recv got EOF from 127.0.0.1:55312 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260810 14:38:18.320783 21725 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/krb5cc' of type FILE
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:18.323359 21725 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:18.325354 21725 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:38:18.345328 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:18.327044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.21.55.65:34513)
0810 14:38:18.327244 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:18.327250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:18.327400 (+ 150us) server_negotiation.cc:407] Connection header received
0810 14:38:18.327538 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:18.327541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:18.327600 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:18.327680 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:18.329727 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.330755 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.332635 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.333084 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.333192 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:18.338000 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:18.338008 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:18.338013 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:18.338070 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:18.341697 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.342463 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.342468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.342470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.342690 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.343440 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.343444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.343447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.343783 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:18.344041 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:18.344628 (+ 587us) server_negotiation.cc:299] Negotiation successful
0810 14:38:18.344978 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 14:38:18.352119 31658 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:18.349404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.21.55.65:34513)
0810 14:38:18.349780 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:18.349787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:18.350184 (+ 397us) server_negotiation.cc:407] Connection header received
0810 14:38:18.350447 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:18.350451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:18.350595 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:18.350910 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:18.351815 (+ 905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55324: BlockingRecv error: recv got EOF from 127.0.0.1:55324 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
W20260810 14:38:18.352161 21725 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786372641103011-21725-0/krb5kdc/krb5cc' of type FILE
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:18.354255 21725 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:18.356189 21725 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:38:18.374655 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:18.358470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55340 (local address 127.21.55.65:34513)
0810 14:38:18.358769 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:18.358779 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:18.359072 (+ 293us) server_negotiation.cc:407] Connection header received
0810 14:38:18.359284 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:18.359294 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:18.359358 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:18.359464 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:18.360391 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.361137 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.362350 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.362865 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.362999 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:18.367326 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:18.367332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:18.367335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:18.367374 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:18.370507 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.371734 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.371738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.371741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.371949 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.372381 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.372385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.372388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.372713 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:18.373287 (+ 574us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:18.373728 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 14:38:18.374002 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260810 14:38:18.376931 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 30950
I20260810 14:38:18.393361 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:18.377957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.21.55.65:34513)
0810 14:38:18.378135 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:18.378139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:18.378714 (+ 575us) server_negotiation.cc:407] Connection header received
0810 14:38:18.378931 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:18.378937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:18.379015 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:18.379131 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:18.380314 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.381047 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.382412 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.382980 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.383138 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:18.386057 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:18.386065 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:18.386069 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:18.386184 (+ 115us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:18.389939 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.390923 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.390931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.390935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.391098 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.391546 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.391554 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.391562 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.391856 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:18.392192 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:18.392518 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 14:38:18.392927 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:38:18.394187 31166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:18.498198 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 30950
I20260810 14:38:18.524374 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 31182
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:38:18.543366 31678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:18.525542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40770 (local address 127.21.55.66:40961)
0810 14:38:18.526148 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:18.526156 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:18.526183 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:38:18.526248 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:18.526254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:18.526472 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:18.526710 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:18.527713 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.528834 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.530305 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.531153 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.531283 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:18.536057 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:18.536073 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:18.536088 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:18.536124 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:18.539595 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.540357 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.540365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.540368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.540507 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.541014 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.541021 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.541024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.541761 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:18.542115 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:18.542608 (+ 493us) server_negotiation.cc:299] Negotiation successful
0810 14:38:18.542861 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":319,"threads_started":1}
I20260810 14:38:18.545094 31403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:18.642097 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 31182
I20260810 14:38:18.668015 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 31423
Aug 10 14:38:18 dist-test-slave-h6n0 krb5kdc[30772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:38:18.686393 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:18.669116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45886 (local address 127.21.55.67:41035)
0810 14:38:18.669701 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:18.669709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:18.669740 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:38:18.669808 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:18.669814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:18.670029 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:18.670210 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:18.671397 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.672403 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.673638 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:18.674197 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:18.674346 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:18.679236 (+ 4890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:18.679259 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:18.679453 (+ 194us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:18.679503 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:18.683371 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.684190 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.684198 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.684200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.684328 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:18.684740 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:18.684745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:18.684750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:18.685148 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:18.685488 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:18.685781 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 14:38:18.686045 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":276,"threads_started":1}
I20260810 14:38:18.688097 31634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:18.775424 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 31423
I20260810 14:38:18.804129 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 30794
I20260810 14:38:18.805768 30901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:18.915259 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 30794
2026-08-10T14:38:18Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4600 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:38:19 dist-test-slave-h6n0 krb5kdc[31694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:38:19 dist-test-slave-h6n0 krb5kdc[31694](info): set up 2 sockets
Aug 10 14:38:19 dist-test-slave-h6n0 krb5kdc[31694](info): commencing operation
krb5kdc: starting...
W20260810 14:38:20.302256 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:38:20 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:38:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:38:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.126.keytab.
Entry for principal oryx/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.126.keytab.
I20260810 14:38:20.571944 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:45213
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.126.keytab
--principal=oryx/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:45213
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:20.907372 31721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:20.907660 31721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:20.907814 31721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:20.914886 31721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:20.914997 31721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:20.915043 31721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:20.915093 31721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:20.915175 31721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:38:20.926308 31721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:45213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.126.keytab
--principal=oryx/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:45213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:20.928053 31721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:20.929438 31721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:20.943002 31731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:20.943069 31734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:20.943043 31732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:20.944324 31721 server_base.cc:1034] running on GCE node
I20260810 14:38:20.945190 31721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:20.946622 31721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:20.947901 31721 hybrid_clock.cc:648] HybridClock initialized: now 1786372700947836 us; error 50 us; skew 500 ppm
I20260810 14:38:20.948280 31721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:20 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372700, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:20.951612 31721 init.cc:377] Logged in from keytab as oryx/127.21.55.126@KRBTEST.COM (short username oryx)
I20260810 14:38:20.953163 31721 webserver.cc:460] Webserver started at http://127.21.55.126:40343/ using document root <none> and password file <none>
I20260810 14:38:20.953856 31721 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:20.954012 31721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:20.954337 31721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:20.957347 31721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "d1b56119bd614ca08ea2f06f6b4b9f40"
format_stamp: "Formatted at 2026-08-10 14:38:20 on dist-test-slave-h6n0"
server_key: "60171c06964b59bf3b0b8d4a05ae7277"
server_key_iv: "d7e995bcad6d55a5c6c31dcb4020c296"
server_key_version: "encryptionkey@0"
I20260810 14:38:20.958159 31721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "d1b56119bd614ca08ea2f06f6b4b9f40"
format_stamp: "Formatted at 2026-08-10 14:38:20 on dist-test-slave-h6n0"
server_key: "60171c06964b59bf3b0b8d4a05ae7277"
server_key_iv: "d7e995bcad6d55a5c6c31dcb4020c296"
server_key_version: "encryptionkey@0"
I20260810 14:38:20.963330 31721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:38:20.966776 31747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:20.968087 31721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:38:20.968281 31721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "d1b56119bd614ca08ea2f06f6b4b9f40"
format_stamp: "Formatted at 2026-08-10 14:38:20 on dist-test-slave-h6n0"
server_key: "60171c06964b59bf3b0b8d4a05ae7277"
server_key_iv: "d7e995bcad6d55a5c6c31dcb4020c296"
server_key_version: "encryptionkey@0"
I20260810 14:38:20.968495 31721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:20.986785 31721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:20.991444 31721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:20.991706 31721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:21.014626 31721 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:45213
I20260810 14:38:21.014710 31831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:45213 every 8 connection(s)
I20260810 14:38:21.016428 31721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:38:21.017748 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 31721
I20260810 14:38:21.018015 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:38:21.018893 21725 external_mini_cluster.cc:1428] Setting key 4a3d362cbc6173951121a7602f84585d
I20260810 14:38:21.022126 31833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:21.037288 31833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40: Bootstrap starting.
I20260810 14:38:21.040753 31833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40: Neither blocks nor log segments found. Creating new log.
Aug 10 14:38:21 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:21.041848 31833 log.cc:824] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:21.045248 31833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40: No bootstrap required, opened a new log
I20260810 14:38:21.050462 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:21.021728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53958 (local address 127.21.55.126:45213)
0810 14:38:21.027179 (+ 5451us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:21.027194 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:21.027680 (+ 486us) server_negotiation.cc:407] Connection header received
0810 14:38:21.030173 (+ 2493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:21.030202 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:21.030751 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:21.031201 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:21.032864 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:21.035603 (+ 2739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:21.036944 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:21.038008 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:21.038183 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:21.042357 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:21.042370 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:21.042395 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:21.042456 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:21.046085 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:21.046993 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:21.046999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:21.047008 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:21.047209 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:21.047648 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:21.047651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:21.047653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:21.048267 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:21.048498 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:21.049031 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 14:38:21.049342 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5144,"thread_start_us":4840,"threads_started":1}
I20260810 14:38:21.055141 31833 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } }
I20260810 14:38:21.055614 31833 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:21.055744 31833 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b56119bd614ca08ea2f06f6b4b9f40, State: Initialized, Role: FOLLOWER
I20260810 14:38:21.056555 31833 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } }
I20260810 14:38:21.057063 31833 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:38:21.057263 31833 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:38:21.057577 31833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:21.060402 31833 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } }
I20260810 14:38:21.060988 31833 leader_election.cc:302] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1b56119bd614ca08ea2f06f6b4b9f40; no voters:
I20260810 14:38:21.061712 31833 leader_election.cc:288] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:38:21.061815 31846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:21.063436 31846 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 1 LEADER]: Becoming Leader. State: Replica: d1b56119bd614ca08ea2f06f6b4b9f40, State: Running, Role: LEADER
I20260810 14:38:21.063977 31846 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } }
I20260810 14:38:21.064128 31833 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:21.066900 31848 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1b56119bd614ca08ea2f06f6b4b9f40. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } }
I20260810 14:38:21.067132 31847 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } }
I20260810 14:38:21.067332 31848 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:21.067400 31847 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:21.068359 31864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:21.074810 31864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:21.085311 31864 catalog_manager.cc:1347] Generated new cluster ID: e7c59bf2e6034f37b02a806d9adc7a34
I20260810 14:38:21.085511 31864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:21.098518 31864 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:38:21.099867 31864 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:21.110505 31864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40: Generated new TSK 0
I20260810 14:38:21.111852 31864 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab.
Entry for principal oryx/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab.
I20260810 14:38:21.181092 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45213
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab
--principal=oryx/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:21.544610 31881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:21.544953 31881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:21.545117 31881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:21.552994 31881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:21.553158 31881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:21.553356 31881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:38:21.564970 31881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab
--principal=oryx/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45213
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:21.566495 31881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:21.568153 31881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:21.582480 31888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:21.582787 31896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:21.583010 31891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:21.583894 31881 server_base.cc:1034] running on GCE node
I20260810 14:38:21.584555 31881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:21.585511 31881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:21.586679 31881 hybrid_clock.cc:648] HybridClock initialized: now 1786372701586633 us; error 31 us; skew 500 ppm
I20260810 14:38:21.587035 31881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:21 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372701, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:21.594875 31881 init.cc:377] Logged in from keytab as oryx/127.21.55.65@KRBTEST.COM (short username oryx)
I20260810 14:38:21.597250 31881 webserver.cc:460] Webserver started at http://127.21.55.65:40277/ using document root <none> and password file <none>
I20260810 14:38:21.598016 31881 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:21.598184 31881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:21.598578 31881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:21.601642 31881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "b4d362b4225743e481b4a4f989a11ebf"
format_stamp: "Formatted at 2026-08-10 14:38:21 on dist-test-slave-h6n0"
server_key: "64f89491fddb2786b165ed28c67e30db"
server_key_iv: "f8209ef34d7bf45e0349b6f89c195014"
server_key_version: "encryptionkey@0"
I20260810 14:38:21.602460 31881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "b4d362b4225743e481b4a4f989a11ebf"
format_stamp: "Formatted at 2026-08-10 14:38:21 on dist-test-slave-h6n0"
server_key: "64f89491fddb2786b165ed28c67e30db"
server_key_iv: "f8209ef34d7bf45e0349b6f89c195014"
server_key_version: "encryptionkey@0"
I20260810 14:38:21.608331 31881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 14:38:21.613783 31912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:21.615726 31881 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 14:38:21.616027 31881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "b4d362b4225743e481b4a4f989a11ebf"
format_stamp: "Formatted at 2026-08-10 14:38:21 on dist-test-slave-h6n0"
server_key: "64f89491fddb2786b165ed28c67e30db"
server_key_iv: "f8209ef34d7bf45e0349b6f89c195014"
server_key_version: "encryptionkey@0"
I20260810 14:38:21.616354 31881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:21.631201 31881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:21.635501 31881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:21.635814 31881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:21.636826 31881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:21.638278 31881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:21.638392 31881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:21.638595 31881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:21.638696 31881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260810 14:38:21.680295 31881 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:34835
I20260810 14:38:21.680341 32093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:34835 every 8 connection(s)
I20260810 14:38:21.681723 31881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:38:21.689385 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 31881
I20260810 14:38:21.689633 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:38:21.690512 21725 external_mini_cluster.cc:1428] Setting key 4ed2bebbd7f10dac9b4fc702ec541af1
Aug 10 14:38:21 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372701, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:21.706070 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:21.684476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:36019 (local address 127.21.55.126:45213)
0810 14:38:21.685407 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:21.685413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:21.686341 (+ 928us) server_negotiation.cc:407] Connection header received
0810 14:38:21.688481 (+ 2140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:21.688486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:21.688558 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:21.688659 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:21.690941 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:21.691878 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:21.694154 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:21.694683 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:21.694792 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:21.699584 (+ 4792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:21.699590 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:21.699593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:21.699627 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:21.702570 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:21.703406 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:21.703413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:21.703417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:21.703585 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:21.704169 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:21.704172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:21.704175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:21.704443 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:21.704572 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:21.705587 (+ 1015us) server_negotiation.cc:299] Negotiation successful
0810 14:38:21.705766 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":351,"threads_started":1}
I20260810 14:38:21.707813 32100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:21.685073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.65:36019)
0810 14:38:21.685954 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:21.686014 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:21.688173 (+ 2159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:21.689005 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:21.689017 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:21.689833 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:21.690738 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:21.690768 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:21.692070 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:21.692078 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:21.693974 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:21.693994 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:21.694924 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:21.694934 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:21.695269 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:21.696110 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:21.696144 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:21.699296 (+ 3152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:21.702751 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:21.702759 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:21.702773 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:21.703252 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:21.703709 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:21.703718 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:21.703725 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:21.704027 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:21.704581 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:21.704593 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:21.705111 (+ 518us) client_negotiation.cc:769] Sending connection context
0810 14:38:21.705446 (+ 335us) client_negotiation.cc:240] Negotiation successful
0810 14:38:21.706117 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":363,"threads_started":1}
I20260810 14:38:21.709817 32094 heartbeater.cc:347] Connected to a master server at 127.21.55.126:45213
I20260810 14:38:21.710253 32094 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:21.711108 32094 heartbeater.cc:511] Master 127.21.55.126:45213 requested a full tablet report, sending...
I20260810 14:38:21.713644 31773 ts_manager.cc:194] Registered new tserver with Master: b4d362b4225743e481b4a4f989a11ebf (127.21.55.65:34835)
I20260810 14:38:21.716150 31773 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:36019
WARNING: no policy specified for oryx/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab.
Entry for principal oryx/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab.
I20260810 14:38:21.754871 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45213
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab
--principal=oryx/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:22.130861 32103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:22.131353 32103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:22.131474 32103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:22.139115 32103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:22.139261 32103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:22.139394 32103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:38:22.150676 32103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab
--principal=oryx/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45213
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:22.152070 32103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:22.153399 32103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:22.168169 32111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:22.169015 32115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:22.169076 32113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:22.172472 32103 server_base.cc:1034] running on GCE node
I20260810 14:38:22.173780 32103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:22.175101 32103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:22.176363 32103 hybrid_clock.cc:648] HybridClock initialized: now 1786372702176327 us; error 34 us; skew 500 ppm
I20260810 14:38:22.176772 32103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:22 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372702, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:22.180526 32103 init.cc:377] Logged in from keytab as oryx/127.21.55.66@KRBTEST.COM (short username oryx)
I20260810 14:38:22.182416 32103 webserver.cc:460] Webserver started at http://127.21.55.66:45983/ using document root <none> and password file <none>
I20260810 14:38:22.183200 32103 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:22.183379 32103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:22.183734 32103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:22.187350 32103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "f6954977858545a6b1890c543c44b792"
format_stamp: "Formatted at 2026-08-10 14:38:22 on dist-test-slave-h6n0"
server_key: "4e09e867d7b4498eedcd2db967985a32"
server_key_iv: "ffd6ba3d1f43c353a964c36a6d95e257"
server_key_version: "encryptionkey@0"
I20260810 14:38:22.188215 32103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "f6954977858545a6b1890c543c44b792"
format_stamp: "Formatted at 2026-08-10 14:38:22 on dist-test-slave-h6n0"
server_key: "4e09e867d7b4498eedcd2db967985a32"
server_key_iv: "ffd6ba3d1f43c353a964c36a6d95e257"
server_key_version: "encryptionkey@0"
I20260810 14:38:22.194408 32103 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260810 14:38:22.199131 32126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:22.200608 32103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:38:22.200815 32103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "f6954977858545a6b1890c543c44b792"
format_stamp: "Formatted at 2026-08-10 14:38:22 on dist-test-slave-h6n0"
server_key: "4e09e867d7b4498eedcd2db967985a32"
server_key_iv: "ffd6ba3d1f43c353a964c36a6d95e257"
server_key_version: "encryptionkey@0"
I20260810 14:38:22.201066 32103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:22.240492 32103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:22.245398 32103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:22.245718 32103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:22.246856 32103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:22.248332 32103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:22.248450 32103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:22.248611 32103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:22.248689 32103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:22.290570 32103 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:40743
I20260810 14:38:22.290627 32318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:40743 every 8 connection(s)
I20260810 14:38:22.291836 32103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:38:22.293783 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 32103
I20260810 14:38:22.293975 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:38:22.294624 21725 external_mini_cluster.cc:1428] Setting key 6423c24dfd9e63a4c7e707934db27018
Aug 10 14:38:22 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372702, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:22.314888 32324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:22.294621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:43575 (local address 127.21.55.126:45213)
0810 14:38:22.295132 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:22.295137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:22.296215 (+ 1078us) server_negotiation.cc:407] Connection header received
0810 14:38:22.298102 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:22.298110 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:22.298200 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:22.298326 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:22.299841 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:22.301024 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:22.303377 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:22.304138 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:22.304302 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:22.308561 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:22.308567 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:22.308570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:22.308604 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:22.311501 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:22.312313 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:22.312324 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:22.312326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:22.312464 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:22.312980 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:22.312986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:22.312989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:22.313265 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:22.313435 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:22.314174 (+ 739us) server_negotiation.cc:299] Negotiation successful
0810 14:38:22.314429 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":295,"threads_started":1}
I20260810 14:38:22.315900 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:22.295187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.66:43575)
0810 14:38:22.296028 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:22.296077 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:22.297836 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:22.298574 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:22.298586 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:22.299113 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:22.299662 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:22.299680 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:22.301189 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:22.301193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:22.303102 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:22.303132 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:22.304504 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:22.304510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:22.304762 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:22.305615 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:22.305639 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:22.308259 (+ 2620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:22.311659 (+ 3400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:22.311671 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:22.311688 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:22.312166 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:22.312602 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:22.312608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:22.312612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:22.312830 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:22.313411 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:22.313419 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:22.313777 (+ 358us) client_negotiation.cc:769] Sending connection context
0810 14:38:22.314074 (+ 297us) client_negotiation.cc:240] Negotiation successful
0810 14:38:22.314703 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":335,"threads_started":1}
I20260810 14:38:22.317631 32319 heartbeater.cc:347] Connected to a master server at 127.21.55.126:45213
I20260810 14:38:22.318130 32319 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:22.319159 32319 heartbeater.cc:511] Master 127.21.55.126:45213 requested a full tablet report, sending...
I20260810 14:38:22.320935 31773 ts_manager.cc:194] Registered new tserver with Master: f6954977858545a6b1890c543c44b792 (127.21.55.66:40743)
WARNING: no policy specified for oryx/127.21.55.67@KRBTEST.COM; defaulting to no policy
I20260810 14:38:22.321973 31773 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:43575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab.
Entry for principal oryx/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab.
I20260810 14:38:22.351774 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45213
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab
--principal=oryx/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:22.712399 32330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:22.712697 32330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:22.712796 32330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 14:38:22.719640 32094 heartbeater.cc:503] Master 127.21.55.126:45213 was elected leader, sending a full tablet report...
W20260810 14:38:22.720860 32330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:22.721027 32330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:22.721372 32330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:38:22.731453 32330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab
--principal=oryx/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45213
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:22.733062 32330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:22.735040 32330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:22.750742 32335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:22.750728 32346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:22.750978 32336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:22.751597 32330 server_base.cc:1034] running on GCE node
I20260810 14:38:22.752247 32330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:22.753198 32330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:22.754441 32330 hybrid_clock.cc:648] HybridClock initialized: now 1786372702754401 us; error 65 us; skew 500 ppm
I20260810 14:38:22.754799 32330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:22 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372702, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:22.757974 32330 init.cc:377] Logged in from keytab as oryx/127.21.55.67@KRBTEST.COM (short username oryx)
I20260810 14:38:22.759460 32330 webserver.cc:460] Webserver started at http://127.21.55.67:39483/ using document root <none> and password file <none>
I20260810 14:38:22.760180 32330 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:22.760334 32330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:22.760651 32330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:22.763594 32330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "4cd99a5acdd841648f05b96fd1a67819"
format_stamp: "Formatted at 2026-08-10 14:38:22 on dist-test-slave-h6n0"
server_key: "0ff28490a87d7e9ac7371e865a1461a5"
server_key_iv: "bf0eaed93f02061d56ae4ba75609d07f"
server_key_version: "encryptionkey@0"
I20260810 14:38:22.764403 32330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "4cd99a5acdd841648f05b96fd1a67819"
format_stamp: "Formatted at 2026-08-10 14:38:22 on dist-test-slave-h6n0"
server_key: "0ff28490a87d7e9ac7371e865a1461a5"
server_key_iv: "bf0eaed93f02061d56ae4ba75609d07f"
server_key_version: "encryptionkey@0"
I20260810 14:38:22.770819 32330 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 14:38:22.775017 32362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:22.776664 32330 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 14:38:22.776954 32330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "4cd99a5acdd841648f05b96fd1a67819"
format_stamp: "Formatted at 2026-08-10 14:38:22 on dist-test-slave-h6n0"
server_key: "0ff28490a87d7e9ac7371e865a1461a5"
server_key_iv: "bf0eaed93f02061d56ae4ba75609d07f"
server_key_version: "encryptionkey@0"
I20260810 14:38:22.777168 32330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:22.820549 32330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:22.825417 32330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:22.825723 32330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:22.826946 32330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:22.828409 32330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:22.828531 32330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:22.828696 32330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:22.828773 32330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:22.872539 32330 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:41079
I20260810 14:38:22.872577 32558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:41079 every 8 connection(s)
I20260810 14:38:22.873991 32330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:38:22.877691 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 32330
I20260810 14:38:22.878001 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:38:22.878830 21725 external_mini_cluster.cc:1428] Setting key 25d8aeba825754b0ed1d34ac703e4b8f
Aug 10 14:38:22 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372702, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:22.901224 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:22.876805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:51425 (local address 127.21.55.126:45213)
0810 14:38:22.877652 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:22.877662 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:22.878894 (+ 1232us) server_negotiation.cc:407] Connection header received
0810 14:38:22.882293 (+ 3399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:22.882298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:22.882411 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:22.882578 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:22.884448 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:22.885455 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:22.887221 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:22.887803 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:22.887983 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:22.893557 (+ 5574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:22.893563 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:22.893567 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:22.893611 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:22.897549 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:22.898401 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:22.898406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:22.898408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:22.898632 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:22.899156 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:22.899160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:22.899162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:22.899445 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:22.899579 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:22.900344 (+ 765us) server_negotiation.cc:299] Negotiation successful
0810 14:38:22.900756 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":496,"threads_started":1}
I20260810 14:38:22.901870 32570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:22.877274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.67:51425)
0810 14:38:22.878569 (+ 1295us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:22.878650 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:22.881911 (+ 3261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:22.882835 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:22.882843 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:22.883601 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:22.884248 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:22.884268 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:22.885633 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:22.885637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:22.886981 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:22.886994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:22.888229 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:22.888237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:22.888722 (+ 485us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:22.889634 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:22.889665 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:22.893252 (+ 3587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:22.897696 (+ 4444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:22.897704 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:22.897717 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:22.898251 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:22.898760 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:22.898764 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:22.898767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:22.899009 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:22.899587 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:22.899594 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:22.899942 (+ 348us) client_negotiation.cc:769] Sending connection context
0810 14:38:22.900196 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 14:38:22.900641 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":441,"threads_started":1}
I20260810 14:38:22.903412 32559 heartbeater.cc:347] Connected to a master server at 127.21.55.126:45213
I20260810 14:38:22.903707 32559 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:22.904469 32559 heartbeater.cc:511] Master 127.21.55.126:45213 requested a full tablet report, sending...
I20260810 14:38:22.906134 31773 ts_manager.cc:194] Registered new tserver with Master: 4cd99a5acdd841648f05b96fd1a67819 (127.21.55.67:41079)
I20260810 14:38:22.907346 31773 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:51425
I20260810 14:38:22.910450 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.55.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.125.keytab.
Entry for principal oryx/127.21.55.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.125.keytab.
I20260810 14:38:22.970360 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.125:35855
--webserver_interface=127.21.55.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.125.keytab
--principal=oryx/127.21.55.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:45213,127.21.55.125:35855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:38:23.325088 32319 heartbeater.cc:503] Master 127.21.55.126:45213 was elected leader, sending a full tablet report...
W20260810 14:38:23.334837 32573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:23.335165 32573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:23.335433 32573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:23.342844 32573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:23.342947 32573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:23.342993 32573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:23.343132 32573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:23.343200 32573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:38:23.354359 32573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:45213,127.21.55.125:35855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.125.keytab
--principal=oryx/127.21.55.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.125:35855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.55.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
W20260810 14:38:23.354868 32573 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.55.126:45213,127.21.55.125:35855'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 14:38:23.355978 32573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:23.357300 32573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:23.369524 32580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:23.369625 32585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:23.369656 32583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:23.371590 32573 server_base.cc:1034] running on GCE node
I20260810 14:38:23.372464 32573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:23.373972 32573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:23.375217 32573 hybrid_clock.cc:648] HybridClock initialized: now 1786372703375159 us; error 27 us; skew 500 ppm
I20260810 14:38:23.375614 32573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:23 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372703, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:23.379151 32573 init.cc:377] Logged in from keytab as oryx/127.21.55.125@KRBTEST.COM (short username oryx)
I20260810 14:38:23.381094 32573 webserver.cc:460] Webserver started at http://127.21.55.125:46195/ using document root <none> and password file <none>
I20260810 14:38:23.381930 32573 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:23.382033 32573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:23.382364 32573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:23.385633 32573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data/instance:
uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3"
format_stamp: "Formatted at 2026-08-10 14:38:23 on dist-test-slave-h6n0"
server_key: "a0f2368998df08bcb09687b6822eb898"
server_key_iv: "fa9b5a606d9ea3dd21941aece9204ecf"
server_key_version: "encryptionkey@0"
I20260810 14:38:23.386499 32573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/wal/instance:
uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3"
format_stamp: "Formatted at 2026-08-10 14:38:23 on dist-test-slave-h6n0"
server_key: "a0f2368998df08bcb09687b6822eb898"
server_key_iv: "fa9b5a606d9ea3dd21941aece9204ecf"
server_key_version: "encryptionkey@0"
I20260810 14:38:23.392277 32573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260810 14:38:23.395845 32599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:23.397197 32573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:38:23.397389 32573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/wal
uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3"
format_stamp: "Formatted at 2026-08-10 14:38:23 on dist-test-slave-h6n0"
server_key: "a0f2368998df08bcb09687b6822eb898"
server_key_iv: "fa9b5a606d9ea3dd21941aece9204ecf"
server_key_version: "encryptionkey@0"
I20260810 14:38:23.397619 32573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:23.422693 32573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:23.428321 32573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:23.428622 32573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:23.450404 32573 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.125:35855
I20260810 14:38:23.450431 32687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.125:35855 every 8 connection(s)
I20260810 14:38:23.451794 32573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
I20260810 14:38:23.456005 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 32573
I20260810 14:38:23.456225 32689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:23.466948 32689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:38:23 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.55.125@KRBTEST.COM
Aug 10 14:38:23 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372703, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.125@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:23.484023 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.458340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.21.55.125:35855)
0810 14:38:23.459136 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:23.459147 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:23.459192 (+ 45us) server_negotiation.cc:407] Connection header received
0810 14:38:23.461019 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:23.461040 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:23.461403 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:23.461786 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:23.463315 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.464746 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.466122 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.467151 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.467349 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.473249 (+ 5900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:23.473260 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:23.473273 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:23.473331 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:23.477758 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.479178 (+ 1420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.479192 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.479207 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.479562 (+ 355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.480441 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.480450 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.480457 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.481241 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:23.481650 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:23.482274 (+ 624us) server_negotiation.cc:299] Negotiation successful
0810 14:38:23.482871 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":343,"threads_started":1}
I20260810 14:38:23.490890 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.470955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.21.55.126:45213)
0810 14:38:23.472584 (+ 1629us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:23.472590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:23.472608 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:38:23.472670 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:23.472674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:23.472753 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:23.472872 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:23.474432 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.475260 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.477276 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.478070 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.478438 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.483594 (+ 5156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:23.483600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:23.483604 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:23.483656 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:23.487959 (+ 4303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.488772 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.488784 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.488787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.488918 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.489299 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.489307 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.489311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.489634 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:23.489795 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:23.490254 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 14:38:23.490478 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1458,"thread_start_us":1327,"threads_started":1}
I20260810 14:38:23.490918 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.470918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:53968)
0810 14:38:23.471735 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:23.471779 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:23.471970 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:23.472949 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:23.472956 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:23.473977 (+ 1021us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:23.474268 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.474277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.475410 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.475418 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.477091 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.477109 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.478434 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.478439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.478664 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.479673 (+ 1009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:23.479701 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:23.483310 (+ 3609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:23.488141 (+ 4831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.488152 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.488159 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.488627 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.489033 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.489038 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.489042 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.489185 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.489769 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:23.489776 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:23.489945 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 14:38:23.490135 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 14:38:23.490476 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":479,"threads_started":1}
I20260810 14:38:23.492906 32689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:38:23 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372703, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.125@KRBTEST.COM for oryx/127.21.55.125@KRBTEST.COM
I20260810 14:38:23.507951 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.493841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:35855 (local address 127.0.0.1:44076)
0810 14:38:23.494023 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:23.494048 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:23.494124 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:23.494482 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:23.494485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:23.494886 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:23.495044 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.495052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.496058 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.496062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.496925 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.496935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.497702 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.497706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.497848 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.498454 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:23.498471 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:23.501199 (+ 2728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:23.504384 (+ 3185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.504389 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.504393 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.505143 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.505574 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.505579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.505583 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.505756 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.506781 (+ 1025us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:23.506785 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:23.507124 (+ 339us) client_negotiation.cc:769] Sending connection context
0810 14:38:23.507306 (+ 182us) client_negotiation.cc:240] Negotiation successful
0810 14:38:23.507643 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 14:38:23.508440 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.493771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.21.55.125:35855)
0810 14:38:23.494056 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:23.494061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:23.494099 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:38:23.494269 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:23.494274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:23.494333 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:23.494427 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:23.495225 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.495909 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.497081 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.497552 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.497679 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.501368 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:23.501373 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:23.501376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:23.501418 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:23.504118 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.505311 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.505316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.505318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.505443 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.505997 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.506003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.506005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.506630 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:23.506753 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:23.507472 (+ 719us) server_negotiation.cc:299] Negotiation successful
0810 14:38:23.507871 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":110,"server-negotiator.queue_time_us":124}
I20260810 14:38:23.513927 32689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: Bootstrap starting.
I20260810 14:38:23.517544 32689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:23.518850 32689 log.cc:824] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:23.521842 32689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: No bootstrap required, opened a new log
I20260810 14:38:23.530243 32689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } }
I20260810 14:38:23.530651 32689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:23.530781 32689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e037c6f6e184e72bd3988d9bd7ca6e3, State: Initialized, Role: FOLLOWER
I20260810 14:38:23.531356 32689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } }
I20260810 14:38:23.532229 32698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } } }
I20260810 14:38:23.532538 32698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:38:23.533228 32689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 14:38:23.542848 32721 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:38:23.543061 32721 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:38:23.548008 32573 master_runner.cc:428] Detected that this master 6e037c6f6e184e72bd3988d9bd7ca6e3 is joining an existing cluster
I20260810 14:38:23.548224 32573 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.55.125:35855
I20260810 14:38:23.577392 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.553113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.21.55.126:45213)
0810 14:38:23.553322 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:23.553331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:23.554136 (+ 805us) server_negotiation.cc:407] Connection header received
0810 14:38:23.554662 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:23.554670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:23.554829 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:23.554916 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:23.556699 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.557421 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.558848 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.559526 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.559626 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.562255 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:23.562261 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:23.562267 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:23.562309 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:23.572582 (+ 10273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.573930 (+ 1348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.573939 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.573945 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.574173 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.575354 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.575364 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.575368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.575861 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:23.576015 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:23.576745 (+ 730us) server_negotiation.cc:299] Negotiation successful
0810 14:38:23.577021 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:38:23.577445 32698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:38:23.577819 32698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } }
WARNING: no policy specified for oryx/127.21.55.124@KRBTEST.COM; defaulting to no policy
I20260810 14:38:23.579613 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.553231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:53984)
0810 14:38:23.554021 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:23.554047 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:23.554315 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:23.555183 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:23.555190 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:23.556086 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:23.556541 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.556553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.557566 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.557573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.558684 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.558700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.559932 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.559939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.560157 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.560871 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:23.560905 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:23.562042 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:23.572921 (+ 10879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.572927 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.572932 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.573711 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.574360 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.574366 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.574370 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.575149 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.576146 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:23.576152 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:23.576390 (+ 238us) client_negotiation.cc:769] Sending connection context
0810 14:38:23.577970 (+ 1580us) client_negotiation.cc:240] Negotiation successful
0810 14:38:23.578976 (+ 1006us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":1568,"threads_started":1}
I20260810 14:38:23.580839 31792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" is_pre_election: true
I20260810 14:38:23.582154 32601 leader_election.cc:302] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 6e037c6f6e184e72bd3988d9bd7ca6e3; no voters: d1b56119bd614ca08ea2f06f6b4b9f40
I20260810 14:38:23.579816 32698 leader_election.cc:288] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1b56119bd614ca08ea2f06f6b4b9f40 (127.21.55.126:45213)
I20260810 14:38:23.582891 32698 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.124@KRBTEST.COM" created.
I20260810 14:38:23.590340 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.554583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:35855 (local address 127.0.0.1:44090)
0810 14:38:23.569771 (+ 15188us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:23.569797 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:23.570020 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:23.570587 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:23.570595 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:23.571380 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:23.571931 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.571948 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.573749 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.573756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.575851 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.575869 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.577504 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.577511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.577704 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.579744 (+ 2040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:23.579766 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:23.582018 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:23.587141 (+ 5123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.587147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.587154 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.587698 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.588182 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.588188 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.588194 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.588362 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.589096 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:23.589103 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:23.589270 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 14:38:23.589421 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 14:38:23.589757 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15009,"thread_start_us":14460,"threads_started":1}
I20260810 14:38:23.591789 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.554221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44090 (local address 127.21.55.125:35855)
0810 14:38:23.554424 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:23.554430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:23.569893 (+ 15463us) server_negotiation.cc:407] Connection header received
0810 14:38:23.570172 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:23.570180 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:23.570265 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:23.570335 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:23.572085 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.573398 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.576012 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.576836 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.576946 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.582200 (+ 5254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:23.582206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:23.582210 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:23.582252 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:23.586980 (+ 4728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.587831 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.587838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.587844 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.588028 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.588474 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.588481 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.588487 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.588868 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:23.589044 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:23.590926 (+ 1882us) server_negotiation.cc:299] Negotiation successful
0810 14:38:23.591238 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 14:38:23 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372700, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.126@KRBTEST.COM for oryx/127.21.55.125@KRBTEST.COM
I20260810 14:38:23.615393 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.589215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44096 (local address 127.21.55.125:35855)
0810 14:38:23.593815 (+ 4600us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:23.593822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:23.594886 (+ 1064us) server_negotiation.cc:407] Connection header received
0810 14:38:23.595285 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:23.595292 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:23.595396 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:23.595496 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:23.597435 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.598726 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.600974 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.601999 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.602138 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.607005 (+ 4867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:23.607015 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:23.607024 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:23.607073 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:23.611965 (+ 4892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.612969 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.613035 (+ 66us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.613056 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.613213 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.613684 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.613687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.613689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.614024 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:23.614183 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:23.614693 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 14:38:23.614979 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4475,"thread_start_us":4772,"threads_started":1}
I20260810 14:38:23.615543 32744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.588825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:35855 (local address 127.0.0.1:44096)
0810 14:38:23.594741 (+ 5916us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:23.594788 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:23.595053 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:23.595807 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:23.595817 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:23.596749 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:23.597110 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.597126 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.598885 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.598893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.600752 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.600778 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.602205 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.602214 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.602517 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.603393 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:23.603421 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:23.606735 (+ 3314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:23.612166 (+ 5431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.612182 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.612191 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.612807 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.613342 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.613349 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.613355 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.613562 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.614158 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:23.614167 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:23.614354 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 14:38:23.614594 (+ 240us) client_negotiation.cc:240] Negotiation successful
0810 14:38:23.615116 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5603,"thread_start_us":5441,"threads_started":1}
I20260810 14:38:23.618005 31773 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.55.125:35855: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3"
member_type: NON_VOTER
last_known_addr {
host: "127.21.55.125"
port: 35855
}
attrs {
promote: true
}
}
dest_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40"
cas_config_opid_index: -1
I20260810 14:38:23.619652 31773 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.124.keytab.
Entry for principal oryx/127.21.55.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.124.keytab.
I20260810 14:38:23.622316 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.124:35125
--webserver_interface=127.21.55.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.124.keytab
--principal=oryx/127.21.55.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:45213,127.21.55.125:35855,127.21.55.124:35125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:38:23.631515 32647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:23.630983 32752 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 6e037c6f6e184e72bd3988d9bd7ca6e3 (127.21.55.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } } }
I20260810 14:38:23.636237 32647 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 FOLLOWER]: Refusing update from remote peer d1b56119bd614ca08ea2f06f6b4b9f40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 14:38:23.637655 32754 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:23.635462 32752 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.55.125:35855
I20260810 14:38:23.640026 32573 master.cc:537] Master@127.21.55.125:35855 shutting down...
W20260810 14:38:23.640991 31752 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260810 14:38:23.645494 31752 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 -> Peer 6e037c6f6e184e72bd3988d9bd7ca6e3 (127.21.55.125:35855): Couldn't send request to peer 6e037c6f6e184e72bd3988d9bd7ca6e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 14:38:23.654973 32751 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } } }
I20260810 14:38:23.655465 32751 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:23.656148 32573 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:23.656652 32573 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:23.656929 32573 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: stopping tablet replica
I20260810 14:38:23.665784 32573 master.cc:552] Master@127.21.55.125:35855 shutdown complete.
I20260810 14:38:23.665980 32573 master_runner.cc:305] Clearing existing system tablet
I20260810 14:38:23.666517 32573 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:23.666740 32573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:23.669930 32573 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 14:38:23.671825 32762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:23.672152 32573 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:23.672598 32573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 14:38:23.672789 32573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/wal
uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3"
format_stamp: "Formatted at 2026-08-10 14:38:23 on dist-test-slave-h6n0"
server_key: "a0f2368998df08bcb09687b6822eb898"
server_key_iv: "fa9b5a606d9ea3dd21941aece9204ecf"
server_key_version: "encryptionkey@0"
I20260810 14:38:23.678340 32573 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:23.687054 32573 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 14:38:23.687251 32573 log.cc:1196] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 14:38:23.687745 32573 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: Deleting consensus metadata
I20260810 14:38:23.688042 32573 master_runner.cc:315] Copying system tablet from 127.21.55.126:45213
I20260810 14:38:23.692458 32573 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: tablet copy: Beginning tablet copy session from remote peer at address 127.21.55.126:45213
I20260810 14:38:23.710033 302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.693420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:53994)
0810 14:38:23.693994 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:23.694018 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:23.694200 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:23.694684 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:23.694691 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:23.695252 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:23.695555 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.695572 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.697042 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.697047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.698510 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:23.698552 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.699552 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.699560 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:23.699814 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.700617 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:23.700645 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:23.702031 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:23.706990 (+ 4959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.706995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.706999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.707487 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.707947 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:23.707952 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:23.707955 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:23.708151 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:23.708880 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:23.708885 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:23.709075 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 14:38:23.709234 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 14:38:23.709557 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":284,"threads_started":1}
I20260810 14:38:23.710044 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:23.693523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53994 (local address 127.21.55.126:45213)
0810 14:38:23.693718 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:23.693727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:23.694136 (+ 409us) server_negotiation.cc:407] Connection header received
0810 14:38:23.694384 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:23.694390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:23.694486 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:23.694617 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:23.695734 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.696867 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.698696 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:23.699356 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:23.699497 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:23.702252 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:23.702259 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:23.702264 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:23.702302 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:23.706819 (+ 4517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.707630 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.707637 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.707641 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.707810 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:23.708287 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:23.708293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:23.708297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:23.708707 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:23.708882 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:23.709376 (+ 494us) server_negotiation.cc:299] Negotiation successful
0810 14:38:23.709628 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 14:38:23.711612 31807 tablet_copy_service.cc:140] P d1b56119bd614ca08ea2f06f6b4b9f40: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6e037c6f6e184e72bd3988d9bd7ca6e3 ({username='oryx', principal='oryx/127.21.55.125@KRBTEST.COM'} at 127.0.0.1:53994)
I20260810 14:38:23.712421 31807 tablet_copy_service.cc:161] P d1b56119bd614ca08ea2f06f6b4b9f40: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6e037c6f6e184e72bd3988d9bd7ca6e3 at {username='oryx', principal='oryx/127.21.55.125@KRBTEST.COM'} at 127.0.0.1:53994: session id = 6e037c6f6e184e72bd3988d9bd7ca6e3-00000000000000000000000000000000
I20260810 14:38:23.717849 31807 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 14:38:23.721127 32573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 14:38:23.730000 32573 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: tablet copy: Starting download of 0 data blocks...
I20260810 14:38:23.730605 32573 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: tablet copy: Starting download of 1 WAL segments...
I20260810 14:38:23.735095 32573 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 14:38:23.743171 31807 tablet_copy_service.cc:342] P d1b56119bd614ca08ea2f06f6b4b9f40: Request end of tablet copy session 6e037c6f6e184e72bd3988d9bd7ca6e3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.55.125@KRBTEST.COM'} at 127.0.0.1:53994
I20260810 14:38:23.743530 31807 tablet_copy_service.cc:433] P d1b56119bd614ca08ea2f06f6b4b9f40: ending tablet copy session 6e037c6f6e184e72bd3988d9bd7ca6e3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6e037c6f6e184e72bd3988d9bd7ca6e3
I20260810 14:38:23.765022 32573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:23.768149 304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:23.768750 305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:23.775504 32573 server_base.cc:1034] running on GCE node
W20260810 14:38:23.778797 308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:23.779309 32573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:23.780015 32573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:23.781283 32573 hybrid_clock.cc:648] HybridClock initialized: now 1786372703781210 us; error 47 us; skew 500 ppm
I20260810 14:38:23.781512 32573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:23 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372703, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:23.785279 32573 init.cc:377] Logged in from keytab as oryx/127.21.55.125@KRBTEST.COM (short username oryx)
I20260810 14:38:23.787614 32573 webserver.cc:460] Webserver started at http://127.21.55.125:35671/ using document root <none> and password file <none>
I20260810 14:38:23.788000 32573 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:23.788110 32573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:23.791358 32573 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 14:38:23.793458 318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:23.793937 32573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 14:38:23.794107 32573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/wal
uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3"
format_stamp: "Formatted at 2026-08-10 14:38:23 on dist-test-slave-h6n0"
server_key: "a0f2368998df08bcb09687b6822eb898"
server_key_iv: "fa9b5a606d9ea3dd21941aece9204ecf"
server_key_version: "encryptionkey@0"
I20260810 14:38:23.794356 32573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:23.807181 32573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:23.811534 32573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:23.841461 403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.125:35855 every 8 connection(s)
I20260810 14:38:23.841526 32573 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.125:35855
I20260810 14:38:23.843161 32573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
I20260810 14:38:23.849682 405 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:38:23.854161 405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: Bootstrap starting.
I20260810 14:38:23.874221 405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:38:23.874931 405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: Bootstrap complete.
I20260810 14:38:23.876101 405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } }
I20260810 14:38:23.876453 405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6e037c6f6e184e72bd3988d9bd7ca6e3, State: Initialized, Role: LEARNER
I20260810 14:38:23.876804 405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } }
I20260810 14:38:23.878319 410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } } }
I20260810 14:38:23.878640 405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:23.878708 410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: This master's current role is: LEARNER
I20260810 14:38:23.887107 435 catalog_manager.cc:1259] Loaded cluster ID: e7c59bf2e6034f37b02a806d9adc7a34
I20260810 14:38:23.887210 435 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: loading cluster ID for follower catalog manager: success
I20260810 14:38:23.890763 435 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: acquiring CA information for follower catalog manager: success
I20260810 14:38:23.893194 435 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:38:23.910748 32559 heartbeater.cc:503] Master 127.21.55.126:45213 was elected leader, sending a full tablet report...
W20260810 14:38:24.064898 32753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:24.065564 32753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:24.065769 32753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:24.073557 32753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:24.073710 32753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:24.073813 32753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:24.073918 32753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:24.073999 32753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:38:24.084148 32753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:45213,127.21.55.125:35855,127.21.55.124:35125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.124.keytab
--principal=oryx/127.21.55.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.124:35125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.55.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:24.085458 32753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:24.086808 32753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:24.099325 455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:24.099342 447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:24.099398 448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:24.100317 32753 server_base.cc:1034] running on GCE node
I20260810 14:38:24.101162 32753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:24.102746 32753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:24.103993 32753 hybrid_clock.cc:648] HybridClock initialized: now 1786372704103935 us; error 40 us; skew 500 ppm
I20260810 14:38:24.104405 32753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:24 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372704, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:24.107836 32753 init.cc:377] Logged in from keytab as oryx/127.21.55.124@KRBTEST.COM (short username oryx)
I20260810 14:38:24.109333 32753 webserver.cc:460] Webserver started at http://127.21.55.124:43861/ using document root <none> and password file <none>
I20260810 14:38:24.110011 32753 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:24.110163 32753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:24.110481 32753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:24.113374 32753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data/instance:
uuid: "bae9e502167c42dda288e1d87c3eed23"
format_stamp: "Formatted at 2026-08-10 14:38:24 on dist-test-slave-h6n0"
server_key: "cecd17366b9e4bc5da8d1cd9c81a4c48"
server_key_iv: "319f58995baf05c123ef480dde8bce3d"
server_key_version: "encryptionkey@0"
I20260810 14:38:24.114133 32753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/wal/instance:
uuid: "bae9e502167c42dda288e1d87c3eed23"
format_stamp: "Formatted at 2026-08-10 14:38:24 on dist-test-slave-h6n0"
server_key: "cecd17366b9e4bc5da8d1cd9c81a4c48"
server_key_iv: "319f58995baf05c123ef480dde8bce3d"
server_key_version: "encryptionkey@0"
I20260810 14:38:24.119246 32753 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:38:24.122848 465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:24.124153 32753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:38:24.124339 32753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/wal
uuid: "bae9e502167c42dda288e1d87c3eed23"
format_stamp: "Formatted at 2026-08-10 14:38:24 on dist-test-slave-h6n0"
server_key: "cecd17366b9e4bc5da8d1cd9c81a4c48"
server_key_iv: "319f58995baf05c123ef480dde8bce3d"
server_key_version: "encryptionkey@0"
I20260810 14:38:24.124536 32753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:24.141700 32753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:24.146970 32753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:24.147290 32753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:24.172163 32753 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.124:35125
I20260810 14:38:24.172233 549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.124:35125 every 8 connection(s)
I20260810 14:38:24.173825 32753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
I20260810 14:38:24.176986 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 32753
I20260810 14:38:24.177659 553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:24.187891 553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:38:24 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.55.124@KRBTEST.COM
Aug 10 14:38:24 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372704, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.124@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:24.207531 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.179301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.21.55.124:35125)
0810 14:38:24.180267 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.180287 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.180341 (+ 54us) server_negotiation.cc:407] Connection header received
0810 14:38:24.182764 (+ 2423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.182792 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.183302 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.183749 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:24.185100 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.187118 (+ 2018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.188440 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.189513 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.189687 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.194846 (+ 5159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:24.194857 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:24.194905 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:24.194987 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:24.198935 (+ 3948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.199749 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.199759 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.199771 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.199986 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.200494 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.200502 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.200514 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.201403 (+ 889us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:24.203507 (+ 2104us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.203881 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.204593 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":367,"threads_started":1}
I20260810 14:38:24.211494 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.191101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53996 (local address 127.21.55.126:45213)
0810 14:38:24.191321 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.191327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.191728 (+ 401us) server_negotiation.cc:407] Connection header received
0810 14:38:24.191935 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.191941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.192025 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.192139 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:24.193309 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.194338 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.196172 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.196859 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.197007 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.201200 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:24.201209 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:24.201216 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:24.201267 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:24.206374 (+ 5107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.207599 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.207608 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.207614 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.207781 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.208351 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.208359 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.208365 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.208838 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:24.209010 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.210655 (+ 1645us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.210979 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:38:24.212893 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.190918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:53996)
0810 14:38:24.191594 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:24.191629 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:24.191807 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:24.192202 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:24.192209 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:24.192822 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:24.193138 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.193155 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.194610 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.194616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.195954 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.195974 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.197139 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.197151 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.197338 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.197944 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:24.197967 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:24.200936 (+ 2969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:24.206626 (+ 5690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.206642 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.206653 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.207414 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.207940 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.207947 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.207954 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.208191 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.209127 (+ 936us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:24.209139 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:24.209402 (+ 263us) client_negotiation.cc:769] Sending connection context
0810 14:38:24.211844 (+ 2442us) client_negotiation.cc:240] Negotiation successful
0810 14:38:24.212311 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":315,"threads_started":1}
I20260810 14:38:24.214354 553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:38:24.219527 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.200234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:35855 (local address 127.0.0.1:44106)
0810 14:38:24.213323 (+ 13089us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:24.213340 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:24.213490 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:24.214162 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:24.214167 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:24.214197 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:24.214408 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.214420 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.216230 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.216237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.217476 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.217488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.218684 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.218690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.218876 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.218932 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 14:38:24.219045 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 14:38:24.219184 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12682,"thread_start_us":12577,"threads_started":1}
I20260810 14:38:24.219702 561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.200516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44106 (local address 127.21.55.125:35855)
0810 14:38:24.212684 (+ 12168us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.212689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.213434 (+ 745us) server_negotiation.cc:407] Connection header received
0810 14:38:24.213628 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.213635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.213727 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.214062 (+ 335us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:24.214671 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.216097 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.217652 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.218453 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.218642 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.218839 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.219074 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.219208 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12046,"thread_start_us":11921,"threads_started":1}
I20260810 14:38:24.220831 365 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260810 14:38:24.221426 410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1b56119bd614ca08ea2f06f6b4b9f40. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } } }
I20260810 14:38:24.221685 410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: This master's current role is: LEARNER
Aug 10 14:38:24 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372704, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.124@KRBTEST.COM for oryx/127.21.55.125@KRBTEST.COM
I20260810 14:38:24.228962 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.215200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:35855 (local address 127.0.0.1:44120)
0810 14:38:24.215398 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:24.215418 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:24.215523 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:24.216344 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:24.216349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:24.216732 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:24.216857 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.216865 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.217975 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.217981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.218988 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.218999 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.219918 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.219926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.220151 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.220814 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:24.220831 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:24.223210 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:24.226495 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.226498 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.226501 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.226959 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.227365 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.227369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.227371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.227525 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.228183 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:24.228189 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:24.228332 (+ 143us) client_negotiation.cc:769] Sending connection context
0810 14:38:24.228455 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 14:38:24.228750 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 14:38:24.229755 563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.215310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.21.55.125:35855)
0810 14:38:24.215870 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.215874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.215891 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:38:24.215926 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.215929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.215978 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.216328 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:24.217033 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.217820 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.219124 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.219580 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.219934 (+ 354us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.223425 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:24.223430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:24.223433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:24.223593 (+ 160us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:24.226364 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.227080 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.227083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.227086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.227225 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.227640 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.227643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.227645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.227955 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:24.228235 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.228933 (+ 698us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.229408 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":328,"threads_started":1}
I20260810 14:38:24.231141 553 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:38:24 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372704, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.124@KRBTEST.COM for oryx/127.21.55.124@KRBTEST.COM
I20260810 14:38:24.247184 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.231756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:35125 (local address 127.0.0.1:59368)
0810 14:38:24.231906 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:24.231920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:24.231996 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:24.232438 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:24.232444 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:24.232935 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:24.233170 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.233181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.234515 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.234519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.235580 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.235597 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.236538 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.236543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.236709 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.237485 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:24.237506 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:24.240360 (+ 2854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:24.243988 (+ 3628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.243993 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.243996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.244478 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.245043 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.245048 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.245050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.245221 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.245925 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:24.245930 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:24.246194 (+ 264us) client_negotiation.cc:769] Sending connection context
0810 14:38:24.246352 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 14:38:24.246829 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 14:38:24.247665 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.231924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59368 (local address 127.21.55.124:35125)
0810 14:38:24.232090 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.232095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.232111 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:38:24.232150 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.232153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.232207 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.232292 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:24.233373 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.234337 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.235793 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.236320 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.236445 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.240603 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:24.240608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:24.240613 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:24.240647 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:24.243806 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.244655 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.244660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.244663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.244852 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.245397 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.245401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.245404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.245750 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:24.245894 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.246472 (+ 578us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.247066 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":217,"server-negotiator.queue_time_us":63}
I20260810 14:38:24.252925 553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: Bootstrap starting.
I20260810 14:38:24.256217 553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:24.257274 553 log.cc:824] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:24.260875 553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: No bootstrap required, opened a new log
I20260810 14:38:24.270452 553 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } }
I20260810 14:38:24.270972 553 raft_consensus.cc:374] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:24.271114 553 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae9e502167c42dda288e1d87c3eed23, State: Initialized, Role: FOLLOWER
I20260810 14:38:24.271871 553 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } }
I20260810 14:38:24.273608 566 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } } }
I20260810 14:38:24.274175 566 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:38:24.274840 553 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 14:38:24.288486 588 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:38:24.288635 588 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:38:24.295905 32753 master_runner.cc:428] Detected that this master bae9e502167c42dda288e1d87c3eed23 is joining an existing cluster
I20260810 14:38:24.296075 32753 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.55.124:35125
I20260810 14:38:24.315894 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 31881
I20260810 14:38:24.318907 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.300363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54004 (local address 127.21.55.126:45213)
0810 14:38:24.300552 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.300559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.301285 (+ 726us) server_negotiation.cc:407] Connection header received
0810 14:38:24.301626 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.301633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.301719 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.301776 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:24.303184 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.304384 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.306775 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.307446 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.307544 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.310419 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:24.310425 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:24.310431 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:24.310469 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:24.314773 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.315596 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.315602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.315606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.315773 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.316302 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.316307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.316310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.316689 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:24.316813 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.318018 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.318333 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 14:38:24.320393 598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.300346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:54004)
0810 14:38:24.301175 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:24.301190 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:24.301453 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:24.301942 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:24.301946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:24.302477 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:24.302991 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.303005 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.304640 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.304645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.306480 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.306491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.307845 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.307851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.308122 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.308850 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:24.308875 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:24.310199 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:24.314954 (+ 4755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.314959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.314963 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.315430 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.315935 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.315939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.315943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.316164 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.317347 (+ 1183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:24.317351 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:24.317622 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 14:38:24.319212 (+ 1590us) client_negotiation.cc:240] Negotiation successful
0810 14:38:24.319690 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"mutex_wait_us":39,"thread_start_us":553,"threads_started":1}
I20260810 14:38:24.321858 599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.300427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:35855 (local address 127.0.0.1:44132)
0810 14:38:24.303140 (+ 2713us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:24.303158 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:24.303323 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:24.303814 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:24.303819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:24.304512 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:24.304932 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.304949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.306477 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.306482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.308303 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.308318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.309574 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.309581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.309779 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.310401 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:24.310422 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:24.311945 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:24.315965 (+ 4020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.315969 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.315973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.316459 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.319758 (+ 3299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.319763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.319766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.319950 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.320684 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:24.320688 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:24.320887 (+ 199us) client_negotiation.cc:769] Sending connection context
0810 14:38:24.321046 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 14:38:24.321372 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2566,"mutex_wait_us":43,"thread_start_us":16722,"threads_started":1}
I20260810 14:38:24.322751 563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.300403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44132 (local address 127.21.55.125:35855)
0810 14:38:24.300564 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.300570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.303223 (+ 2653us) server_negotiation.cc:407] Connection header received
0810 14:38:24.303477 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.303482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.303558 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.303626 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:24.305207 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.306252 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.308485 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.309166 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.309268 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.312168 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:24.312175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:24.312180 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:24.312221 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:24.315824 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.316593 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.316598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.316602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.316756 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.320068 (+ 3312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.320074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.320078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.320451 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:24.320620 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.322188 (+ 1568us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.322421 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 14:38:24 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.55.65@KRBTEST.COM
Aug 10 14:38:24 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372700, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.126@KRBTEST.COM for oryx/127.21.55.124@KRBTEST.COM
I20260810 14:38:24.341646 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.301465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.21.55.124:35125)
0810 14:38:24.301635 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.301640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.317535 (+ 15895us) server_negotiation.cc:407] Connection header received
0810 14:38:24.317717 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.317721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.317785 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.317868 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:24.319490 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.320615 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.329595 (+ 8980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.330471 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.330631 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.333329 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:24.333335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:24.333341 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:24.333382 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:24.338231 (+ 4849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.339118 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.339123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.339126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.339299 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.339756 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.339763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.339769 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.340180 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:24.340368 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.340962 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.341227 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 14:38:24.343061 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.301386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:35125 (local address 127.0.0.1:59382)
0810 14:38:24.317421 (+ 16035us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:24.317435 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:24.317602 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:24.318017 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:24.318022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:24.318701 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:24.319333 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.319349 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.321066 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.321070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.329333 (+ 8263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.329372 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.331015 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.331024 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.331255 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.331983 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:24.332006 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:24.333135 (+ 1129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:24.338363 (+ 5228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.338370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.338374 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.338999 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.339467 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.339472 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.339478 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.339642 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.340446 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:24.340452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:24.340642 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 14:38:24.342139 (+ 1497us) client_negotiation.cc:240] Negotiation successful
0810 14:38:24.342463 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15898,"thread_start_us":1370,"threads_started":1}
I20260810 14:38:24.345743 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.326410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:35125 (local address 127.0.0.1:59396)
0810 14:38:24.326818 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:24.326836 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:24.326935 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:24.327503 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:24.327508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:24.328504 (+ 996us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:24.328705 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.328717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.331268 (+ 2551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.331273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.332615 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.332631 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.333683 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.333689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.334313 (+ 624us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.335309 (+ 996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:24.335338 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:24.338403 (+ 3065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:24.343098 (+ 4695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.343106 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.343112 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.343601 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.344233 (+ 632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.344239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.344243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.344393 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.344909 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:24.344915 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:24.345088 (+ 173us) client_negotiation.cc:769] Sending connection context
0810 14:38:24.345235 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 14:38:24.345510 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288}
I20260810 14:38:24.346761 606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.326462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59396 (local address 127.21.55.124:35125)
0810 14:38:24.327189 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.327193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.327210 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:38:24.327277 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.327280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.327334 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.327462 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:24.328931 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.331132 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.332788 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.333502 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.336830 (+ 3328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.338712 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:24.338718 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:24.338723 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:24.338762 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:24.342900 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.343879 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.343884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.343887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.344052 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.344512 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.344517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.344520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.344784 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:24.344938 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.346208 (+ 1270us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.346419 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":473,"threads_started":1}
I20260810 14:38:24.347438 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.317330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47840 (local address 127.21.55.65:34835)
0810 14:38:24.320465 (+ 3135us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.320474 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.320516 (+ 42us) server_negotiation.cc:407] Connection header received
0810 14:38:24.320591 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.320600 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.320884 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.322900 (+ 2016us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:24.323190 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.324782 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.327111 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.327992 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.335153 (+ 7161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.335492 (+ 339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:24.335508 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:24.335527 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:24.335567 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:24.341710 (+ 6143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.342772 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.342782 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.342786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.342980 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.343487 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.343494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.343497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.343983 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:24.346430 (+ 2447us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.346637 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.346951 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2968,"thread_start_us":5624,"threads_started":1}
I20260810 14:38:24.348415 31773 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.55.124:35125: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bae9e502167c42dda288e1d87c3eed23"
member_type: NON_VOTER
last_known_addr {
host: "127.21.55.124"
port: 35125
}
attrs {
promote: true
}
}
dest_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40"
cas_config_opid_index: 5
I20260810 14:38:24.349444 31773 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } }
I20260810 14:38:24.349668 32087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:24.351807 612 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bae9e502167c42dda288e1d87c3eed23 (127.21.55.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:24.352630 365 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 LEARNER]: Refusing update from remote peer d1b56119bd614ca08ea2f06f6b4b9f40: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 14:38:24.353152 507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:24.354843 559 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 14:38:24.355897 612 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.55.124:35125
I20260810 14:38:24.356356 612 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:24.356693 612 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:24.366693 365 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bae9e502167c42dda288e1d87c3eed23 (127.21.55.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:24.371951 410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: NON_VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: true } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:24.372324 410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: This master's current role is: LEARNER
I20260810 14:38:24.373097 612 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40: attempting to promote NON_VOTER 6e037c6f6e184e72bd3988d9bd7ca6e3 to VOTER
I20260810 14:38:24.374068 612 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } }
I20260810 14:38:24.375155 507 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 FOLLOWER]: Refusing update from remote peer d1b56119bd614ca08ea2f06f6b4b9f40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 14:38:24.375741 32753 master.cc:537] Master@127.21.55.124:35125 shutting down...
I20260810 14:38:24.382714 365 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 LEARNER]: Refusing update from remote peer d1b56119bd614ca08ea2f06f6b4b9f40: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 14:38:24.384050 608 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 14:38:24.393778 559 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6e037c6f6e184e72bd3988d9bd7ca6e3 (127.21.55.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:24.395279 364 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 6e037c6f6e184e72bd3988d9bd7ca6e3 (127.21.55.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:24.398342 608 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:24.398730 608 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:24.399137 559 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:24.399391 559 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:24.400336 410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:24.400645 410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 14:38:24.401077 31751 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 -> Peer bae9e502167c42dda288e1d87c3eed23 (127.21.55.124:35125): Couldn't send request to peer bae9e502167c42dda288e1d87c3eed23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
W20260810 14:38:24.401139 472 tablet.cc:2369] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:38:24.404192 32753 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:24.404636 32753 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:24.404865 32753 tablet_replica.cc:330] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: stopping tablet replica
I20260810 14:38:24.423462 32753 master.cc:552] Master@127.21.55.124:35125 shutdown complete.
I20260810 14:38:24.423573 32753 master_runner.cc:305] Clearing existing system tablet
I20260810 14:38:24.424012 32753 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:24.424127 32753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:24.428009 32753 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 14:38:24.429765 620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:24.430217 32753 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:24.430636 32753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 14:38:24.430809 32753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/wal
uuid: "bae9e502167c42dda288e1d87c3eed23"
format_stamp: "Formatted at 2026-08-10 14:38:24 on dist-test-slave-h6n0"
server_key: "cecd17366b9e4bc5da8d1cd9c81a4c48"
server_key_iv: "319f58995baf05c123ef480dde8bce3d"
server_key_version: "encryptionkey@0"
I20260810 14:38:24.435932 32753 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:24.443138 32753 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 14:38:24.443445 32753 log.cc:1196] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 14:38:24.444075 32753 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: Deleting consensus metadata
I20260810 14:38:24.444446 32753 master_runner.cc:315] Copying system tablet from 127.21.55.126:45213
I20260810 14:38:24.448091 32753 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: tablet copy: Beginning tablet copy session from remote peer at address 127.21.55.126:45213
I20260810 14:38:24.466004 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.449183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.21.55.126:45213)
0810 14:38:24.449452 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.449460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.451115 (+ 1655us) server_negotiation.cc:407] Connection header received
0810 14:38:24.451327 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.451333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.451424 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.451532 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:24.452679 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.453676 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.455607 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.456235 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.456370 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.458827 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:24.458834 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:24.458839 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:24.458875 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:24.462403 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.463272 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.463277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.463282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.463445 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:24.463928 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:24.463933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:24.463936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:24.464285 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:24.464444 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.465302 (+ 858us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.465566 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260810 14:38:24.467083 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.449122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:54014)
0810 14:38:24.450970 (+ 1848us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:24.451000 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:24.451177 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:24.451586 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:24.451593 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:24.452184 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:24.452505 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.452519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.453839 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.453846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.455437 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.455454 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.456423 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.456430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.456617 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.457386 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:24.457414 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:24.458600 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:24.462575 (+ 3975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.462581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.462586 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.463119 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.463569 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:24.463574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:24.463578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:24.463774 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:24.464412 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:24.464418 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:24.464642 (+ 224us) client_negotiation.cc:769] Sending connection context
0810 14:38:24.464813 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 14:38:24.466405 (+ 1592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1707,"thread_start_us":1565,"threads_started":1}
I20260810 14:38:24.467234 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 31881
I20260810 14:38:24.467747 31807 tablet_copy_service.cc:140] P d1b56119bd614ca08ea2f06f6b4b9f40: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bae9e502167c42dda288e1d87c3eed23 ({username='oryx', principal='oryx/127.21.55.124@KRBTEST.COM'} at 127.0.0.1:54014)
I20260810 14:38:24.467963 31807 tablet_copy_service.cc:161] P d1b56119bd614ca08ea2f06f6b4b9f40: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bae9e502167c42dda288e1d87c3eed23 at {username='oryx', principal='oryx/127.21.55.124@KRBTEST.COM'} at 127.0.0.1:54014: session id = bae9e502167c42dda288e1d87c3eed23-00000000000000000000000000000000
I20260810 14:38:24.472537 31807 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 14:38:24.477587 32753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 14:38:24.486382 32753 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: tablet copy: Starting download of 0 data blocks...
I20260810 14:38:24.486763 32753 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: tablet copy: Starting download of 1 WAL segments...
I20260810 14:38:24.489499 32753 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 14:38:24.493861 31807 tablet_copy_service.cc:342] P d1b56119bd614ca08ea2f06f6b4b9f40: Request end of tablet copy session bae9e502167c42dda288e1d87c3eed23-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.55.124@KRBTEST.COM'} at 127.0.0.1:54014
I20260810 14:38:24.494019 31807 tablet_copy_service.cc:433] P d1b56119bd614ca08ea2f06f6b4b9f40: ending tablet copy session bae9e502167c42dda288e1d87c3eed23-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bae9e502167c42dda288e1d87c3eed23
I20260810 14:38:24.501495 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:34835
--local_ip_for_outbound_sockets=127.21.55.65
--tserver_master_addrs=127.21.55.126:45213
--webserver_port=40277
--webserver_interface=127.21.55.65
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab
--principal=oryx/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.55.126:45213,127.21.55.125:35855
--tserver_master_addrs=127.21.55.126:45213,127.21.55.125:35855,127.21.55.124:35125 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:38:24.508103 32753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:24.511363 629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:24.512719 630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:24.528429 640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:24.530705 32753 server_base.cc:1034] running on GCE node
I20260810 14:38:24.531150 32753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:24.531898 32753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:24.533110 32753 hybrid_clock.cc:648] HybridClock initialized: now 1786372704533071 us; error 32 us; skew 500 ppm
I20260810 14:38:24.533371 32753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:24 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372704, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:24.536478 32753 init.cc:377] Logged in from keytab as oryx/127.21.55.124@KRBTEST.COM (short username oryx)
I20260810 14:38:24.540112 32753 webserver.cc:460] Webserver started at http://127.21.55.124:46561/ using document root <none> and password file <none>
I20260810 14:38:24.540614 32753 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:24.540774 32753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:24.544881 32753 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:38:24.547890 653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:24.548851 32753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 14:38:24.549026 32753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/wal
uuid: "bae9e502167c42dda288e1d87c3eed23"
format_stamp: "Formatted at 2026-08-10 14:38:24 on dist-test-slave-h6n0"
server_key: "cecd17366b9e4bc5da8d1cd9c81a4c48"
server_key_iv: "319f58995baf05c123ef480dde8bce3d"
server_key_version: "encryptionkey@0"
I20260810 14:38:24.549239 32753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:24.564293 32753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:24.569542 32753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:24.615763 32753 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.124:35125
I20260810 14:38:24.615819 742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.124:35125 every 8 connection(s)
I20260810 14:38:24.617199 32753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
I20260810 14:38:24.622563 743 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:38:24.626717 743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: Bootstrap starting.
I20260810 14:38:24.644721 743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:38:24.645247 743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: Bootstrap complete.
I20260810 14:38:24.646430 743 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } }
I20260810 14:38:24.646766 743 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bae9e502167c42dda288e1d87c3eed23, State: Initialized, Role: LEARNER
I20260810 14:38:24.647059 743 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } }
I20260810 14:38:24.648327 743 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:24.648011 752 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:24.648622 752 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: This master's current role is: LEARNER
I20260810 14:38:24.676653 771 catalog_manager.cc:1259] Loaded cluster ID: e7c59bf2e6034f37b02a806d9adc7a34
I20260810 14:38:24.676805 771 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: loading cluster ID for follower catalog manager: success
I20260810 14:38:24.679445 771 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: acquiring CA information for follower catalog manager: success
I20260810 14:38:24.681450 771 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:38:24.836127 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.827481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:35125 (local address 127.0.0.1:59404)
0810 14:38:24.827684 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:24.827709 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:24.827881 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:24.828920 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:24.828926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:24.828960 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:24.829189 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.829220 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.831341 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.831348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.833263 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:24.833285 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.835167 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.835176 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:24.835425 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.835467 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 14:38:24.835584 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 14:38:24.835758 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 14:38:24.836170 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:24.827676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59404 (local address 127.21.55.124:35125)
0810 14:38:24.828494 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:24.828501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:24.828528 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:38:24.828613 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:24.828620 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:24.828717 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:24.828846 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:24.829432 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.831176 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.833475 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:24.834923 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:24.835088 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:24.835356 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:24.835634 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 14:38:24.835769 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":503,"threads_started":1}
I20260810 14:38:24.837419 700 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 14:38:24.838075 752 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1b56119bd614ca08ea2f06f6b4b9f40. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:24.838414 752 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: This master's current role is: LEARNER
W20260810 14:38:24.973291 636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:24.973635 636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:24.973794 636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:24.983773 636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:24.983927 636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:24.984174 636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:38:25.001595 636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab
--principal=oryx/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:34835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=40277
--tserver_master_addrs=127.21.55.126:45213,127.21.55.125:35855,127.21.55.124:35125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:25.003216 636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:25.004626 636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:25.018657 779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:25.018550 782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:25.018833 784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:25.024293 636 server_base.cc:1034] running on GCE node
I20260810 14:38:25.024966 636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:25.026055 636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:25.027292 636 hybrid_clock.cc:648] HybridClock initialized: now 1786372705027234 us; error 50 us; skew 500 ppm
I20260810 14:38:25.027628 636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:25 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372705, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:25.032648 636 init.cc:377] Logged in from keytab as oryx/127.21.55.65@KRBTEST.COM (short username oryx)
I20260810 14:38:25.034447 636 webserver.cc:460] Webserver started at http://127.21.55.65:40277/ using document root <none> and password file <none>
I20260810 14:38:25.035266 636 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:25.035423 636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:25.041206 636 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:38:25.044528 796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:25.046784 636 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:38:25.047115 636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "b4d362b4225743e481b4a4f989a11ebf"
format_stamp: "Formatted at 2026-08-10 14:38:21 on dist-test-slave-h6n0"
server_key: "64f89491fddb2786b165ed28c67e30db"
server_key_iv: "f8209ef34d7bf45e0349b6f89c195014"
server_key_version: "encryptionkey@0"
I20260810 14:38:25.048692 636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:25.068749 636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:25.073865 636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:25.074178 636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:25.075242 636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:25.076690 636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:25.076828 636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:25.076960 636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:25.077065 636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:25.116335 636 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:34835
I20260810 14:38:25.116463 991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:34835 every 8 connection(s)
I20260810 14:38:25.117841 636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:38:25.124078 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 636
I20260810 14:38:25.124481 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 32103
Aug 10 14:38:25 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372705, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.124@KRBTEST.COM
Aug 10 14:38:25 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372705, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
Aug 10 14:38:25 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372705, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.125@KRBTEST.COM
Aug 10 14:38:25 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.55.66@KRBTEST.COM
I20260810 14:38:25.146641 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.123616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:34067 (local address 127.21.55.124:35125)
0810 14:38:25.123840 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:25.123847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:25.124180 (+ 333us) server_negotiation.cc:407] Connection header received
0810 14:38:25.127011 (+ 2831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:25.127018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:25.127134 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:25.127234 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:25.128927 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.129983 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.132654 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.133313 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.133431 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.138474 (+ 5043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:25.138479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:25.138483 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:25.138516 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:25.142563 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.143576 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.143583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.143588 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.143761 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.144366 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.144371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.144373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.144763 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:25.144904 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:25.145962 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0810 14:38:25.146239 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:38:25.146672 1000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.121149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:41833 (local address 127.21.55.125:35855)
0810 14:38:25.122291 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:25.122302 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:25.122995 (+ 693us) server_negotiation.cc:407] Connection header received
0810 14:38:25.126317 (+ 3322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:25.126322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:25.126420 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:25.126495 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:25.129398 (+ 2903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.130724 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.132970 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.133610 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.133716 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.139625 (+ 5909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:25.139633 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:25.139640 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:25.139684 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:25.143486 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.144373 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.144378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.144382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.144513 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.145006 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.145011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.145014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.145359 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:25.145494 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:25.146051 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 14:38:25.146259 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":962,"thread_start_us":825,"threads_started":1}
I20260810 14:38:25.148483 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.123487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:35125 (local address 127.21.55.65:34067)
0810 14:38:25.124052 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:25.124078 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:25.126866 (+ 2788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:25.127526 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:25.127530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:25.128321 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:25.128756 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.128768 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.130155 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.130159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.132496 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.132512 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.133728 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.133735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.134009 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.134934 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:25.134969 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:25.138236 (+ 3267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:25.142736 (+ 4500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.142746 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.142763 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.143406 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.143922 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.143926 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.143930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.144222 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.144997 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:25.145004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:25.145446 (+ 442us) client_negotiation.cc:769] Sending connection context
0810 14:38:25.147312 (+ 1866us) client_negotiation.cc:240] Negotiation successful
0810 14:38:25.147594 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":120,"spinlock_wait_cycles":23296,"thread_start_us":303,"threads_started":1}
I20260810 14:38:25.148723 1003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.121747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:35855 (local address 127.21.55.65:41833)
0810 14:38:25.122795 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:25.122853 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:25.125929 (+ 3076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:25.126745 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:25.126748 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:25.128918 (+ 2170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:25.129196 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.129209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.130855 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.130860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.132821 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.132832 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.133996 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.134003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.134218 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.135836 (+ 1618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:25.135860 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:25.139359 (+ 3499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:25.143637 (+ 4278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.143644 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.143647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.144248 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.144666 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.144671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.144674 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.144875 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.145577 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:25.145580 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:25.145761 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 14:38:25.146958 (+ 1197us) client_negotiation.cc:240] Negotiation successful
0810 14:38:25.147274 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":450,"spinlock_wait_cycles":4864,"thread_start_us":497,"threads_started":1}
I20260810 14:38:25.149428 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.121010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:44761 (local address 127.21.55.126:45213)
0810 14:38:25.122730 (+ 1720us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:25.122739 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:25.124014 (+ 1275us) server_negotiation.cc:407] Connection header received
0810 14:38:25.125886 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:25.125893 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:25.126084 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:25.126206 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:25.128539 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.129748 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.133425 (+ 3677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.134081 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.134805 (+ 724us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.139060 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:25.139069 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:25.139075 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:25.139119 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:25.143217 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.144348 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.144355 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.144359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.144503 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.145113 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.145122 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.145128 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.145540 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:25.148503 (+ 2963us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:25.148717 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 14:38:25.149012 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1585,"thread_start_us":1482,"threads_started":1}
I20260810 14:38:25.148945 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.121748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.65:44761)
0810 14:38:25.123131 (+ 1383us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:25.123150 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:25.125552 (+ 2402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:25.126520 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:25.126556 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:25.127549 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:25.128306 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.128333 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.130001 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.130006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.133261 (+ 3255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.133274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.134292 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.134297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.134485 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.135077 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:25.135101 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:25.138811 (+ 3710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:25.143378 (+ 4567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.143383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.143387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.144204 (+ 817us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.144641 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.144647 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.144650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.144842 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.145708 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:25.145713 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:25.145892 (+ 179us) client_negotiation.cc:769] Sending connection context
0810 14:38:25.146076 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 14:38:25.146647 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1171,"mutex_wait_us":1861,"thread_start_us":814,"threads_started":1}
I20260810 14:38:25.151022 996 heartbeater.cc:347] Connected to a master server at 127.21.55.125:35855
I20260810 14:38:25.151114 995 heartbeater.cc:347] Connected to a master server at 127.21.55.126:45213
I20260810 14:38:25.151383 995 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:25.151461 993 heartbeater.cc:347] Connected to a master server at 127.21.55.124:35125
I20260810 14:38:25.151583 993 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:25.152326 993 heartbeater.cc:511] Master 127.21.55.124:35125 requested a full tablet report, sending...
I20260810 14:38:25.152637 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.125543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.21.55.66:40743)
0810 14:38:25.127428 (+ 1885us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:25.127436 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:25.127462 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:38:25.127544 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:25.127551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:25.127802 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:25.128242 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:25.130493 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.131633 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.133328 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.134157 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.134878 (+ 721us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.142732 (+ 7854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:25.142749 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:25.142765 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:25.142811 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:25.147501 (+ 4690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.148570 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.148579 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.148582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.148769 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.150178 (+ 1409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.150183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.150186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.151012 (+ 826us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:25.151336 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:25.151972 (+ 636us) server_negotiation.cc:299] Negotiation successful
0810 14:38:25.152238 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1722,"thread_start_us":1579,"threads_started":1}
I20260810 14:38:25.151392 996 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:25.153728 996 heartbeater.cc:511] Master 127.21.55.125:35855 requested a full tablet report, sending...
I20260810 14:38:25.154497 32312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:25.154887 995 heartbeater.cc:511] Master 127.21.55.126:45213 requested a full tablet report, sending...
I20260810 14:38:25.155086 686 ts_manager.cc:194] Registered new tserver with Master: b4d362b4225743e481b4a4f989a11ebf (127.21.55.65:34835)
I20260810 14:38:25.156356 349 ts_manager.cc:194] Registered new tserver with Master: b4d362b4225743e481b4a4f989a11ebf (127.21.55.65:34835)
I20260810 14:38:25.157325 31773 ts_manager.cc:194] Re-registered known tserver with Master: b4d362b4225743e481b4a4f989a11ebf (127.21.55.65:34835)
I20260810 14:38:25.158908 31773 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:44761
I20260810 14:38:25.232756 559 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: NON_VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: true } } }
I20260810 14:38:25.233222 559 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:25.235373 1020 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40: attempting to promote NON_VOTER bae9e502167c42dda288e1d87c3eed23 to VOTER
I20260810 14:38:25.236210 1020 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } }
I20260810 14:38:25.239115 700 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 LEARNER]: Refusing update from remote peer d1b56119bd614ca08ea2f06f6b4b9f40: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 14:38:25.239184 364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 FOLLOWER]: Refusing update from remote peer d1b56119bd614ca08ea2f06f6b4b9f40: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 14:38:25.239777 1022 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260810 14:38:25.240649 1020 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 14:38:25.244647 559 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bae9e502167c42dda288e1d87c3eed23 (127.21.55.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:25.246682 700 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bae9e502167c42dda288e1d87c3eed23 (127.21.55.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:25.247318 364 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bae9e502167c42dda288e1d87c3eed23 (127.21.55.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:25.248873 559 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:25.249186 559 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:25.249459 559 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:25.249724 559 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:25.249958 559 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:25.250226 559 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:25.252291 752 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:25.252578 752 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:38:25.255168 1025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:25.255632 1025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:38:25.256183 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 32103
I20260810 14:38:25.286688 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:40743
--local_ip_for_outbound_sockets=127.21.55.66
--tserver_master_addrs=127.21.55.126:45213
--webserver_port=45983
--webserver_interface=127.21.55.66
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab
--principal=oryx/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.55.126:45213,127.21.55.125:35855
--tserver_master_addrs=127.21.55.126:45213,127.21.55.125:35855,127.21.55.124:35125 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:25.637290 1026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:25.637591 1026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:25.637769 1026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:25.645203 1026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:25.645316 1026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:25.645536 1026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:38:25.656584 1026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab
--principal=oryx/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:40743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=45983
--tserver_master_addrs=127.21.55.126:45213,127.21.55.125:35855,127.21.55.124:35125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:25.658010 1026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:25.659497 1026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:25.674417 1035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:25.674597 1040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:25.674599 1036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:25.675829 1026 server_base.cc:1034] running on GCE node
I20260810 14:38:25.676457 1026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:25.677405 1026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:25.678638 1026 hybrid_clock.cc:648] HybridClock initialized: now 1786372705678582 us; error 66 us; skew 500 ppm
I20260810 14:38:25.678988 1026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:25 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372705, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:25.683091 1026 init.cc:377] Logged in from keytab as oryx/127.21.55.66@KRBTEST.COM (short username oryx)
I20260810 14:38:25.684847 1026 webserver.cc:460] Webserver started at http://127.21.55.66:45983/ using document root <none> and password file <none>
I20260810 14:38:25.685626 1026 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:25.685804 1026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:25.691769 1026 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:38:25.695007 1049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:25.696547 1026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:38:25.696732 1026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "f6954977858545a6b1890c543c44b792"
format_stamp: "Formatted at 2026-08-10 14:38:22 on dist-test-slave-h6n0"
server_key: "4e09e867d7b4498eedcd2db967985a32"
server_key_iv: "ffd6ba3d1f43c353a964c36a6d95e257"
server_key_version: "encryptionkey@0"
I20260810 14:38:25.697804 1026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:25.736868 1026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:25.741313 1026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:25.741626 1026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:25.742674 1026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:25.744052 1026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:25.744149 1026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:25.744225 1026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:25.744282 1026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:25.780668 1026 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:40743
I20260810 14:38:25.780738 1236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:40743 every 8 connection(s)
I20260810 14:38:25.782480 1026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:38:25.792676 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 1026
I20260810 14:38:25.793035 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 32330
Aug 10 14:38:25 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372705, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.125@KRBTEST.COM
Aug 10 14:38:25 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372705, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.124@KRBTEST.COM
Aug 10 14:38:25 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372705, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
Aug 10 14:38:25 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.55.67@KRBTEST.COM
I20260810 14:38:25.837080 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.787015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:36473 (local address 127.21.55.125:35855)
0810 14:38:25.803194 (+ 16179us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:25.803205 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:25.804339 (+ 1134us) server_negotiation.cc:407] Connection header received
0810 14:38:25.804444 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:25.804453 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:25.804574 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:25.804719 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:25.808474 (+ 3755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.809977 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.813559 (+ 3582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.814229 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.814342 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.821240 (+ 6898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:25.821247 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:25.821253 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:25.821297 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:25.826057 (+ 4760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.827287 (+ 1230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.827293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.827296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.827438 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.828011 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.828016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.828019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.828546 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:25.828707 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:25.836170 (+ 7463us) server_negotiation.cc:299] Negotiation successful
0810 14:38:25.836518 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15955,"thread_start_us":15818,"threads_started":1}
I20260810 14:38:25.838136 1260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.810765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:35125 (local address 127.21.55.66:45671)
0810 14:38:25.811415 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:25.811434 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:25.811570 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:25.816343 (+ 4773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:25.816348 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:25.816849 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:25.817336 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.817354 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.819578 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.819584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.821322 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.821339 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.822520 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.822948 (+ 428us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.823256 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.823868 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:25.823906 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:25.827006 (+ 3100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:25.833695 (+ 6689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.833703 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.833710 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.834281 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.834800 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.834805 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.834809 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.834997 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.835698 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:25.835703 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:25.835870 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 14:38:25.836046 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 14:38:25.836624 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":2,"thread_start_us":566,"threads_started":1}
I20260810 14:38:25.838200 1246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.786253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:35855 (local address 127.21.55.66:36473)
0810 14:38:25.803924 (+ 17671us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:25.803956 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:25.804177 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:25.805355 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:25.805375 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:25.806682 (+ 1307us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:25.808188 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.808227 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.811209 (+ 2982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.811216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.813360 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.813385 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.814733 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.814743 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.815992 (+ 1249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.817176 (+ 1184us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:25.817219 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:25.820984 (+ 3765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:25.826243 (+ 5259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.826253 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.826276 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.827153 (+ 877us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.827557 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.827565 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.827570 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.827852 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.830800 (+ 2948us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:25.830815 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:25.831749 (+ 934us) client_negotiation.cc:769] Sending connection context
0810 14:38:25.837351 (+ 5602us) client_negotiation.cc:240] Negotiation successful
0810 14:38:25.837677 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17400,"mutex_wait_us":38,"spinlock_wait_cycles":28416,"thread_start_us":4980,"threads_started":1}
I20260810 14:38:25.838917 1258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.794243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42516 (local address 127.21.55.67:41079)
0810 14:38:25.811445 (+ 17202us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:25.811454 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:25.811485 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:38:25.811557 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:25.811569 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:25.811928 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:25.812146 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:25.814634 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.816153 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.818396 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.819318 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.819458 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.829341 (+ 9883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:25.829362 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:25.829385 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:25.829462 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:25.834296 (+ 4834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.835277 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.835287 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.835290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.835457 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.835936 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.835942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.835945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.836375 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:25.836691 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:25.838163 (+ 1472us) server_negotiation.cc:299] Negotiation successful
0810 14:38:25.838451 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16998,"thread_start_us":18145,"threads_started":1}
I20260810 14:38:25.841207 32552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:25.842163 1244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.785397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:45429 (local address 127.21.55.126:45213)
0810 14:38:25.814501 (+ 29104us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:25.814511 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:25.814557 (+ 46us) server_negotiation.cc:407] Connection header received
0810 14:38:25.814638 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:25.814645 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:25.814748 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:25.814899 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:25.815813 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.816843 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.818935 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.819583 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.819709 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.828868 (+ 9159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:25.828877 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:25.828884 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:25.829229 (+ 345us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:25.835681 (+ 6452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.839443 (+ 3762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.839450 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.839454 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.839626 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.840637 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.840643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.840647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.840987 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:25.841147 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:25.841594 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 14:38:25.841818 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28932,"thread_start_us":29624,"threads_started":1}
I20260810 14:38:25.842463 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.786253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.66:45429)
0810 14:38:25.791903 (+ 5650us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:25.791970 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:25.796209 (+ 4239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:25.814909 (+ 18700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:25.814914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:25.815396 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:25.815648 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.815660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.816998 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.817006 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.818778 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:25.818793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.819789 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.819798 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:25.820032 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.820580 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:25.820601 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:25.827903 (+ 7302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:25.838772 (+ 10869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.838776 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.838780 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.839297 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.840304 (+ 1007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:25.840309 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:25.840313 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:25.840492 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:25.841115 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:25.841124 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:25.841292 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 14:38:25.841451 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 14:38:25.841865 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5337,"mutex_wait_us":48,"thread_start_us":17035,"threads_started":1}
I20260810 14:38:25.843379 1241 heartbeater.cc:347] Connected to a master server at 127.21.55.126:45213
I20260810 14:38:25.843724 1241 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:25.844050 1259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:25.804562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:45671 (local address 127.21.55.124:35125)
0810 14:38:25.815921 (+ 11359us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:25.815929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:25.815952 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:38:25.816007 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:25.816012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:25.816083 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:25.817515 (+ 1432us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:25.817676 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.819434 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.821494 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:25.822152 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:25.823602 (+ 1450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:25.828154 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:25.828160 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:25.828164 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:25.828203 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:25.833529 (+ 5326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.834458 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.834464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.834469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.834659 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:25.835154 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:25.835162 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:25.835167 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:25.835559 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:25.843111 (+ 7552us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:25.843412 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 14:38:25.843648 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11216,"thread_start_us":11570,"threads_started":1}
I20260810 14:38:25.844646 1241 heartbeater.cc:511] Master 127.21.55.126:45213 requested a full tablet report, sending...
I20260810 14:38:25.844848 1237 heartbeater.cc:347] Connected to a master server at 127.21.55.124:35125
I20260810 14:38:25.844980 1237 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:25.845408 1237 heartbeater.cc:511] Master 127.21.55.124:35125 requested a full tablet report, sending...
I20260810 14:38:25.846184 31773 ts_manager.cc:194] Re-registered known tserver with Master: f6954977858545a6b1890c543c44b792 (127.21.55.66:40743)
I20260810 14:38:25.846557 686 ts_manager.cc:194] Registered new tserver with Master: f6954977858545a6b1890c543c44b792 (127.21.55.66:40743)
I20260810 14:38:25.846602 1242 heartbeater.cc:347] Connected to a master server at 127.21.55.125:35855
I20260810 14:38:25.846720 1242 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:25.847237 1242 heartbeater.cc:511] Master 127.21.55.125:35855 requested a full tablet report, sending...
I20260810 14:38:25.847580 31773 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:45429
I20260810 14:38:25.851212 349 ts_manager.cc:194] Registered new tserver with Master: f6954977858545a6b1890c543c44b792 (127.21.55.66:40743)
I20260810 14:38:26.004475 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 32330
I20260810 14:38:26.033126 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:41079
--local_ip_for_outbound_sockets=127.21.55.67
--tserver_master_addrs=127.21.55.126:45213
--webserver_port=39483
--webserver_interface=127.21.55.67
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab
--principal=oryx/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:35299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.55.126:45213,127.21.55.125:35855
--tserver_master_addrs=127.21.55.126:45213,127.21.55.125:35855,127.21.55.124:35125 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:38:26.162395 995 heartbeater.cc:503] Master 127.21.55.126:45213 was elected leader, sending a full tablet report...
W20260810 14:38:26.392696 1264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:26.393002 1264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:26.393163 1264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:26.400388 1264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:26.400545 1264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:26.400774 1264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:38:26.411692 1264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:35299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab
--principal=oryx/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:41079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=39483
--tserver_master_addrs=127.21.55.126:45213,127.21.55.125:35855,127.21.55.124:35125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:26.413029 1264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:26.414299 1264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:26.427733 1278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:26.427729 1275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:26.428396 1274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:26.428707 1264 server_base.cc:1034] running on GCE node
I20260810 14:38:26.429342 1264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:26.430269 1264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:26.431529 1264 hybrid_clock.cc:648] HybridClock initialized: now 1786372706431471 us; error 83 us; skew 500 ppm
I20260810 14:38:26.431855 1264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:26 dist-test-slave-h6n0 krb5kdc[31694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372706, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:26.435545 1264 init.cc:377] Logged in from keytab as oryx/127.21.55.67@KRBTEST.COM (short username oryx)
I20260810 14:38:26.437106 1264 webserver.cc:460] Webserver started at http://127.21.55.67:39483/ using document root <none> and password file <none>
I20260810 14:38:26.437789 1264 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:26.437959 1264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:26.443825 1264 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:38:26.446796 1290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:26.448946 1264 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 14:38:26.449304 1264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "4cd99a5acdd841648f05b96fd1a67819"
format_stamp: "Formatted at 2026-08-10 14:38:22 on dist-test-slave-h6n0"
server_key: "0ff28490a87d7e9ac7371e865a1461a5"
server_key_iv: "bf0eaed93f02061d56ae4ba75609d07f"
server_key_version: "encryptionkey@0"
I20260810 14:38:26.450961 1264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:26.472806 1264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:26.477222 1264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:26.477516 1264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:26.478585 1264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:26.480068 1264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:26.480199 1264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:26.480330 1264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:26.480420 1264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:26.519217 1264 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:41079
I20260810 14:38:26.519311 1481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:41079 every 8 connection(s)
I20260810 14:38:26.520685 1264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:38:26.524237 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 1264
Aug 10 14:38:26 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372706, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
Aug 10 14:38:26 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372706, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.125@KRBTEST.COM
Aug 10 14:38:26 dist-test-slave-h6n0 krb5kdc[31694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372706, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.124@KRBTEST.COM
I20260810 14:38:26.573333 1492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.523733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:48593 (local address 127.21.55.125:35855)
0810 14:38:26.545982 (+ 22249us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.545987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.546000 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:38:26.546055 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.546058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.546118 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.546252 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:26.549164 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.550487 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.553159 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.553816 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.553916 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.560664 (+ 6748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:26.560672 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:26.560677 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:26.560716 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:26.564742 (+ 4026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.566518 (+ 1776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.566559 (+ 41us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.566565 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.566897 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.567572 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.567577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.567580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.567950 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:26.568115 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.572608 (+ 4493us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.572835 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22133,"thread_start_us":22033,"threads_started":1}
I20260810 14:38:26.575608 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.524877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:35855 (local address 127.21.55.67:48593)
0810 14:38:26.539007 (+ 14130us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:26.539047 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:26.544498 (+ 5451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:26.546552 (+ 2054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:26.546568 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:26.547728 (+ 1160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:26.548969 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.548982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.550709 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.550715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.552970 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.552992 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.554273 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.554284 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.554624 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.555724 (+ 1100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:26.555764 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:26.560456 (+ 4692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:26.564907 (+ 4451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:26.564917 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:26.564935 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:26.566204 (+ 1269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:26.567074 (+ 870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:26.567081 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:26.567084 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:26.567382 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:26.568945 (+ 1563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:26.568956 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:26.569993 (+ 1037us) client_negotiation.cc:769] Sending connection context
0810 14:38:26.570465 (+ 472us) client_negotiation.cc:240] Negotiation successful
0810 14:38:26.571521 (+ 1056us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13964,"mutex_wait_us":272,"thread_start_us":15767,"threads_started":1}
I20260810 14:38:26.579864 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.524388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.67:56905)
0810 14:38:26.536455 (+ 12067us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:26.536537 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:26.546491 (+ 9954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:26.547075 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:26.547081 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:26.547728 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:26.548925 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.548954 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.550709 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.550715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.553253 (+ 2538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.553265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.554553 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.554561 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.554730 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.555796 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:26.555834 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:26.559646 (+ 3812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:26.568184 (+ 8538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:26.568190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:26.568193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:26.568750 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:26.573447 (+ 4697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:26.573454 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:26.573460 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:26.573845 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:26.578459 (+ 4614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:26.578464 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:26.578724 (+ 260us) client_negotiation.cc:769] Sending connection context
0810 14:38:26.578882 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 14:38:26.579251 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11646,"mutex_wait_us":535,"spinlock_wait_cycles":15616,"thread_start_us":11383,"threads_started":1}
I20260810 14:38:26.580462 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.523733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:56905 (local address 127.21.55.126:45213)
0810 14:38:26.545554 (+ 21821us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.545564 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.545593 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:38:26.546682 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.546687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.546770 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.546876 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:26.549167 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.550487 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.553394 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.554053 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.554149 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.560028 (+ 5879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:26.560036 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:26.560041 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:26.560084 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:26.568082 (+ 7998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.573153 (+ 5071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.573160 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.573164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.573337 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.577840 (+ 4503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.577843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.577845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.578307 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:26.578476 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.579805 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.580082 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21626,"thread_start_us":22631,"threads_started":1}
I20260810 14:38:26.583781 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.542955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:35125 (local address 127.21.55.67:42805)
0810 14:38:26.554148 (+ 11193us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:26.554179 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:26.554336 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:26.555739 (+ 1403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:26.555747 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:26.556817 (+ 1070us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:26.557401 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.557428 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.558452 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.558459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.560135 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.560153 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.561948 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.561954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.562133 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.563946 (+ 1813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:26.563980 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:26.572652 (+ 8672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:26.578650 (+ 5998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:26.578655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:26.578661 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:26.580246 (+ 1585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:26.580730 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:26.580735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:26.580738 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:26.580909 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:26.582505 (+ 1596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:26.582513 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:26.582777 (+ 264us) client_negotiation.cc:769] Sending connection context
0810 14:38:26.582936 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 14:38:26.583229 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11011,"thread_start_us":11517,"threads_started":1}
I20260810 14:38:26.584348 1488 heartbeater.cc:347] Connected to a master server at 127.21.55.126:45213
I20260810 14:38:26.584995 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.537393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:42805 (local address 127.21.55.124:35125)
0810 14:38:26.555119 (+ 17726us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.555126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.555146 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:38:26.555207 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.555215 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.555298 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.555406 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:26.557593 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.558306 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.560312 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.560958 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.561091 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.574149 (+ 13058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:26.574157 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:26.574162 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:26.574341 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:26.577728 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.580388 (+ 2660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.580394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.580397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.580580 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.581049 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.581054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.581058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.582196 (+ 1138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:26.582430 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.584167 (+ 1737us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.584466 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17589,"thread_start_us":18144,"threads_started":1}
I20260810 14:38:26.585080 1488 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:26.586668 1488 heartbeater.cc:511] Master 127.21.55.126:45213 requested a full tablet report, sending...
I20260810 14:38:26.588884 31773 ts_manager.cc:194] Re-registered known tserver with Master: 4cd99a5acdd841648f05b96fd1a67819 (127.21.55.67:41079)
I20260810 14:38:26.590119 31773 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:56905
I20260810 14:38:26.591770 1489 heartbeater.cc:347] Connected to a master server at 127.21.55.125:35855
I20260810 14:38:26.591954 1489 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:26.594115 1489 heartbeater.cc:511] Master 127.21.55.125:35855 requested a full tablet report, sending...
I20260810 14:38:26.595237 1487 heartbeater.cc:347] Connected to a master server at 127.21.55.124:35125
I20260810 14:38:26.595435 1487 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:26.596269 1487 heartbeater.cc:511] Master 127.21.55.124:35125 requested a full tablet report, sending...
I20260810 14:38:26.596271 349 ts_manager.cc:194] Registered new tserver with Master: 4cd99a5acdd841648f05b96fd1a67819 (127.21.55.67:41079)
I20260810 14:38:26.597540 686 ts_manager.cc:194] Registered new tserver with Master: 4cd99a5acdd841648f05b96fd1a67819 (127.21.55.67:41079)
I20260810 14:38:26.607802 1508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.561469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54024 (local address 127.21.55.126:45213)
0810 14:38:26.561986 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.561992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.576209 (+ 14217us) server_negotiation.cc:407] Connection header received
0810 14:38:26.576276 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.576282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.576373 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.576497 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:26.585383 (+ 8886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.587287 (+ 1904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.591107 (+ 3820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.591781 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.591971 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.600145 (+ 8174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:26.600154 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:26.600162 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:26.600205 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:26.603676 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.604484 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.604492 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.604497 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.604662 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.605963 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.605970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.605974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.606382 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:26.606672 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.607093 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.607342 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":283,"threads_started":1}
I20260810 14:38:26.611006 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.563842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59416 (local address 127.21.55.124:35125)
0810 14:38:26.585272 (+ 21430us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.585280 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.590044 (+ 4764us) server_negotiation.cc:407] Connection header received
0810 14:38:26.591920 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.591924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.592023 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.592095 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:26.595436 (+ 3341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.596420 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.599595 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.600263 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.600399 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.603828 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:26.603833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:26.603837 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:26.603869 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:26.607466 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.608385 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.608393 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.608398 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.608548 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.609044 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.609054 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.609061 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.609445 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:26.609694 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.610194 (+ 500us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.610628 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21302,"thread_start_us":27457,"threads_started":1}
I20260810 14:38:26.613899 1492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.568495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44136 (local address 127.21.55.125:35855)
0810 14:38:26.574403 (+ 5908us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.574411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.595342 (+ 20931us) server_negotiation.cc:407] Connection header received
0810 14:38:26.596977 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.596982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.597084 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.597189 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:26.599893 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.600769 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.602961 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.603642 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.603789 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.607114 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:26.607120 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:26.607124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:26.607164 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:26.610897 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.611773 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.611778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.611783 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.611927 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.612368 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.612373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.612375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.612666 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:26.612964 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.613288 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.613547 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5674,"thread_start_us":21766,"threads_started":1}
I20260810 14:38:26.612337 31773 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:38:26.620077 31773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:38:26.669888 1531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.647084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:34835 (local address 127.0.0.1:47852)
0810 14:38:26.659494 (+ 12410us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:26.659509 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:26.659609 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:26.660782 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:26.660790 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:26.660821 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:26.661391 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.661405 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.663455 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.663460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.666039 (+ 2579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.666055 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.668194 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.668202 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.668831 (+ 629us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.668895 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 14:38:26.669065 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 14:38:26.669257 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12299,"mutex_wait_us":60,"thread_start_us":11811,"threads_started":1}
I20260810 14:38:26.669994 1533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.642736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.21.55.66:40743)
0810 14:38:26.661664 (+ 18928us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.661675 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.661703 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:38:26.661799 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.661806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.662005 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.662200 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:26.662810 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.664347 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.667130 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.668529 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.668812 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.669020 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.669496 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.669672 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18735,"thread_start_us":18637,"threads_started":1}
I20260810 14:38:26.669916 1536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.642598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:41079 (local address 127.0.0.1:42530)
0810 14:38:26.659135 (+ 16537us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:26.659151 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:26.659260 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:26.660506 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:26.660511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:26.660536 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:26.661041 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.661055 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.663712 (+ 2657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.663717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.665890 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.665903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.668200 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.668213 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.669134 (+ 921us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.669163 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 14:38:26.669237 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 14:38:26.669339 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16437,"mutex_wait_us":605,"thread_start_us":16633,"threads_started":1}
I20260810 14:38:26.670084 1532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.642122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:40743 (local address 127.0.0.1:59070)
0810 14:38:26.658934 (+ 16812us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:26.658961 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:26.659135 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:26.662219 (+ 3084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:26.662224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:26.662250 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:26.662651 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.662664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.664509 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.664514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.666969 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.666981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.668996 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.669006 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.669205 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.669238 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:38:26.669435 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 14:38:26.669624 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16598,"mutex_wait_us":3,"thread_start_us":16498,"threads_started":1}
I20260810 14:38:26.670933 1534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.643004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42530 (local address 127.21.55.67:41079)
0810 14:38:26.655397 (+ 12393us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.655405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.659289 (+ 3884us) server_negotiation.cc:407] Connection header received
0810 14:38:26.659460 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.659473 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.660232 (+ 759us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.660500 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:26.661634 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.663123 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.666052 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.667649 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.667785 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.668082 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.670340 (+ 2258us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.670553 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11988,"thread_start_us":11725,"threads_started":1}
I20260810 14:38:26.671307 1535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.646691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47852 (local address 127.21.55.65:34835)
0810 14:38:26.660104 (+ 13413us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.660113 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.660137 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:38:26.660260 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.660266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.660544 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.661077 (+ 533us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:26.661612 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.663313 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.666181 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.667643 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.667780 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.668002 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.670490 (+ 2488us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.670956 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13201,"thread_start_us":13068,"threads_started":1}
I20260810 14:38:26.674199 1366 tablet_service.cc:1467] Processing CreateTablet for tablet ba1fb063ae1e4567bdf66446ad27e08b (DEFAULT_TABLE table=test-table [id=423e27bdf6044a66857f5c9735ab30ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:26.674427 887 tablet_service.cc:1467] Processing CreateTablet for tablet ba1fb063ae1e4567bdf66446ad27e08b (DEFAULT_TABLE table=test-table [id=423e27bdf6044a66857f5c9735ab30ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:26.674427 1126 tablet_service.cc:1467] Processing CreateTablet for tablet ba1fb063ae1e4567bdf66446ad27e08b (DEFAULT_TABLE table=test-table [id=423e27bdf6044a66857f5c9735ab30ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:26.675729 1366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1fb063ae1e4567bdf66446ad27e08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:26.675771 887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1fb063ae1e4567bdf66446ad27e08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:26.676213 1126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1fb063ae1e4567bdf66446ad27e08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:26.689680 1538 tablet_bootstrap.cc:492] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: Bootstrap starting.
I20260810 14:38:26.690902 1541 tablet_bootstrap.cc:492] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf: Bootstrap starting.
I20260810 14:38:26.693871 1538 tablet_bootstrap.cc:654] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:26.694339 1541 tablet_bootstrap.cc:654] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:26.695241 1538 log.cc:824] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:26.695712 1541 log.cc:824] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:26.696256 1542 tablet_bootstrap.cc:492] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: Bootstrap starting.
I20260810 14:38:26.699654 1542 tablet_bootstrap.cc:654] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:26.700693 1542 log.cc:824] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:26.702677 1541 tablet_bootstrap.cc:492] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf: No bootstrap required, opened a new log
I20260810 14:38:26.702822 1538 tablet_bootstrap.cc:492] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: No bootstrap required, opened a new log
I20260810 14:38:26.703035 1541 ts_tablet_manager.cc:1397] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260810 14:38:26.703111 1538 ts_tablet_manager.cc:1397] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260810 14:38:26.705081 1542 tablet_bootstrap.cc:492] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: No bootstrap required, opened a new log
I20260810 14:38:26.705350 1542 ts_tablet_manager.cc:1397] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260810 14:38:26.710848 1538 raft_consensus.cc:348] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:26.711217 1538 raft_consensus.cc:374] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:26.711342 1538 raft_consensus.cc:729] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd99a5acdd841648f05b96fd1a67819, State: Initialized, Role: FOLLOWER
I20260810 14:38:26.711970 1538 consensus_queue.cc:260] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:26.713275 1488 heartbeater.cc:503] Master 127.21.55.126:45213 was elected leader, sending a full tablet report...
I20260810 14:38:26.714205 1541 raft_consensus.cc:348] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:26.714656 1541 raft_consensus.cc:374] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:26.714745 1541 raft_consensus.cc:729] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d362b4225743e481b4a4f989a11ebf, State: Initialized, Role: FOLLOWER
I20260810 14:38:26.715364 1541 consensus_queue.cc:260] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:26.716622 1542 raft_consensus.cc:348] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:26.717193 1542 raft_consensus.cc:374] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:26.717324 1542 raft_consensus.cc:729] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6954977858545a6b1890c543c44b792, State: Initialized, Role: FOLLOWER
I20260810 14:38:26.717967 1538 ts_tablet_manager.cc:1428] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: Time spent starting tablet: real 0.015s user 0.010s sys 0.000s
I20260810 14:38:26.718182 1542 consensus_queue.cc:260] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:26.721496 1541 ts_tablet_manager.cc:1428] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf: Time spent starting tablet: real 0.018s user 0.010s sys 0.007s
I20260810 14:38:26.725984 1542 ts_tablet_manager.cc:1428] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: Time spent starting tablet: real 0.020s user 0.013s sys 0.009s
I20260810 14:38:26.726267 1241 heartbeater.cc:503] Master 127.21.55.126:45213 was elected leader, sending a full tablet report...
I20260810 14:38:26.732205 1549 raft_consensus.cc:482] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:38:26.732427 1549 raft_consensus.cc:504] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:26.733943 1549 leader_election.cc:288] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6954977858545a6b1890c543c44b792 (127.21.55.66:40743), b4d362b4225743e481b4a4f989a11ebf (127.21.55.65:34835)
I20260810 14:38:26.739730 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.734131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:34835 (local address 127.21.55.67:45771)
0810 14:38:26.734397 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:26.734417 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:26.734566 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:26.734877 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:26.734890 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:26.734918 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:26.735127 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.735136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.736222 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.736228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.737599 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.737611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.738825 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.738833 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.739088 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.739142 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 14:38:26.739241 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 14:38:26.739379 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":45}
I20260810 14:38:26.739872 1535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.734337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:45771 (local address 127.21.55.65:34835)
0810 14:38:26.734584 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.734590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.734610 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:38:26.734676 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.734681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.734760 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.734859 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:26.735274 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.736073 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.737750 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.738618 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.738773 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.738857 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.739495 (+ 638us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.739633 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 14:38:26.740373 1533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.734177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:57325 (local address 127.21.55.66:40743)
0810 14:38:26.734370 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.734377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.734675 (+ 298us) server_negotiation.cc:407] Connection header received
0810 14:38:26.734874 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.734878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.734928 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.735033 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:26.735387 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.736170 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.737789 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.738959 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.739150 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.739246 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.740031 (+ 785us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.740209 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:38:26.740370 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.734131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:40743 (local address 127.21.55.67:57325)
0810 14:38:26.734579 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:26.734602 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:26.734742 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:26.735049 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:26.735053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:26.735079 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:26.735259 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.735266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.736330 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.736335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.737635 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:26.737647 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.739146 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.739153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:26.739357 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.739389 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:38:26.739487 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 14:38:26.739618 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":150}
I20260810 14:38:26.740749 920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1fb063ae1e4567bdf66446ad27e08b" candidate_uuid: "4cd99a5acdd841648f05b96fd1a67819" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d362b4225743e481b4a4f989a11ebf" is_pre_election: true
I20260810 14:38:26.741274 920 raft_consensus.cc:2454] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd99a5acdd841648f05b96fd1a67819 in term 0.
I20260810 14:38:26.741437 1160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1fb063ae1e4567bdf66446ad27e08b" candidate_uuid: "4cd99a5acdd841648f05b96fd1a67819" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6954977858545a6b1890c543c44b792" is_pre_election: true
I20260810 14:38:26.741940 1160 raft_consensus.cc:2454] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cd99a5acdd841648f05b96fd1a67819 in term 0.
I20260810 14:38:26.741955 1294 leader_election.cc:302] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd99a5acdd841648f05b96fd1a67819, b4d362b4225743e481b4a4f989a11ebf; no voters:
I20260810 14:38:26.742367 1549 raft_consensus.cc:2781] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:38:26.742571 1549 raft_consensus.cc:482] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:38:26.742697 1549 raft_consensus.cc:3037] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:26.744899 1549 raft_consensus.cc:504] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:26.745486 1549 leader_election.cc:288] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [CANDIDATE]: Term 1 election: Requested vote from peers f6954977858545a6b1890c543c44b792 (127.21.55.66:40743), b4d362b4225743e481b4a4f989a11ebf (127.21.55.65:34835)
I20260810 14:38:26.745901 1160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1fb063ae1e4567bdf66446ad27e08b" candidate_uuid: "4cd99a5acdd841648f05b96fd1a67819" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6954977858545a6b1890c543c44b792"
I20260810 14:38:26.746099 920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1fb063ae1e4567bdf66446ad27e08b" candidate_uuid: "4cd99a5acdd841648f05b96fd1a67819" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d362b4225743e481b4a4f989a11ebf"
I20260810 14:38:26.746265 1160 raft_consensus.cc:3037] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:26.746428 920 raft_consensus.cc:3037] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:26.748821 1160 raft_consensus.cc:2454] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd99a5acdd841648f05b96fd1a67819 in term 1.
I20260810 14:38:26.748997 920 raft_consensus.cc:2454] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cd99a5acdd841648f05b96fd1a67819 in term 1.
I20260810 14:38:26.749310 1291 leader_election.cc:302] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd99a5acdd841648f05b96fd1a67819, f6954977858545a6b1890c543c44b792; no voters:
I20260810 14:38:26.749645 1549 raft_consensus.cc:2781] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:26.750250 1549 raft_consensus.cc:686] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 1 LEADER]: Becoming Leader. State: Replica: 4cd99a5acdd841648f05b96fd1a67819, State: Running, Role: LEADER
I20260810 14:38:26.750825 1549 consensus_queue.cc:237] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:26.756906 31773 catalog_manager.cc:5495] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cd99a5acdd841648f05b96fd1a67819 (127.21.55.67). New cstate: current_term: 1 leader_uuid: "4cd99a5acdd841648f05b96fd1a67819" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } health_report { overall_health: UNKNOWN } } }
W20260810 14:38:26.773605 1490 tablet.cc:2369] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:38:26.791582 1243 tablet.cc:2369] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:38:26.797286 1534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.791606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.21.55.67:41079)
0810 14:38:26.791789 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.791795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.791885 (+ 90us) server_negotiation.cc:407] Connection header received
0810 14:38:26.792128 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.792136 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.792258 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.792402 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:38:26.792781 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.793480 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.795234 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.795698 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.795809 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.796275 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:38:26.796718 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:38:26.796823 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.796925 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.796987 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:38:26.805433 920 raft_consensus.cc:1261] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf [term 1 FOLLOWER]: Refusing update from remote peer 4cd99a5acdd841648f05b96fd1a67819: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:26.805429 1160 raft_consensus.cc:1261] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792 [term 1 FOLLOWER]: Refusing update from remote peer 4cd99a5acdd841648f05b96fd1a67819: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:26.806416 1553 consensus_queue.cc:1035] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:26.806831 1549 consensus_queue.cc:1035] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:26.838212 31773 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:26.838439 31773 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:26.846612 31773 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1b56119bd614ca08ea2f06f6b4b9f40: Sending DeleteTablet for 3 replicas of tablet ba1fb063ae1e4567bdf66446ad27e08b
I20260810 14:38:26.848091 1126 tablet_service.cc:1514] Processing DeleteTablet for tablet ba1fb063ae1e4567bdf66446ad27e08b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:26 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:59070
I20260810 14:38:26.848549 887 tablet_service.cc:1514] Processing DeleteTablet for tablet ba1fb063ae1e4567bdf66446ad27e08b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:26 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:47852
I20260810 14:38:26.849144 1578 tablet_replica.cc:330] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: stopping tablet replica
I20260810 14:38:26.849349 1366 tablet_service.cc:1514] Processing DeleteTablet for tablet ba1fb063ae1e4567bdf66446ad27e08b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:26 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:42530
I20260810 14:38:26.849617 1578 raft_consensus.cc:2229] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:26.850199 1578 raft_consensus.cc:2258] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:26.854998 1583 tablet_replica.cc:330] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: stopping tablet replica
I20260810 14:38:26.855602 1583 raft_consensus.cc:2229] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:38:26.856365 1583 raft_consensus.cc:2258] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:26.859030 1578 ts_tablet_manager.cc:1905] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:26.866276 1583 ts_tablet_manager.cc:1905] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:26.867411 1578 ts_tablet_manager.cc:1918] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:26.867674 1578 log.cc:1196] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/ba1fb063ae1e4567bdf66446ad27e08b
I20260810 14:38:26.868428 1578 ts_tablet_manager.cc:1939] T ba1fb063ae1e4567bdf66446ad27e08b P f6954977858545a6b1890c543c44b792: Deleting consensus metadata
I20260810 14:38:26.870424 31750 catalog_manager.cc:4841] TS f6954977858545a6b1890c543c44b792 (127.21.55.66:40743): tablet ba1fb063ae1e4567bdf66446ad27e08b (table test-table [id=423e27bdf6044a66857f5c9735ab30ca]) successfully deleted
I20260810 14:38:26.872254 1581 tablet_replica.cc:330] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf: stopping tablet replica
I20260810 14:38:26.873236 1581 raft_consensus.cc:2229] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:26.873901 1581 raft_consensus.cc:2258] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:26.875725 1581 ts_tablet_manager.cc:1905] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:26.881654 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 31721
I20260810 14:38:26.882862 31823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:26.882992 1581 ts_tablet_manager.cc:1918] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:26.883162 1581 log.cc:1196] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/ba1fb063ae1e4567bdf66446ad27e08b
I20260810 14:38:26.883837 1581 ts_tablet_manager.cc:1939] T ba1fb063ae1e4567bdf66446ad27e08b P b4d362b4225743e481b4a4f989a11ebf: Deleting consensus metadata
I20260810 14:38:26.886037 31751 catalog_manager.cc:4841] TS b4d362b4225743e481b4a4f989a11ebf (127.21.55.65:34835): tablet ba1fb063ae1e4567bdf66446ad27e08b (table test-table [id=423e27bdf6044a66857f5c9735ab30ca]) successfully deleted
I20260810 14:38:26.897079 1583 ts_tablet_manager.cc:1918] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:26.897302 1583 log.cc:1196] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/ba1fb063ae1e4567bdf66446ad27e08b
I20260810 14:38:26.898025 1583 ts_tablet_manager.cc:1939] T ba1fb063ae1e4567bdf66446ad27e08b P 4cd99a5acdd841648f05b96fd1a67819: Deleting consensus metadata
I20260810 14:38:26.898517 1492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.850631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44144 (local address 127.21.55.125:35855)
0810 14:38:26.850841 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.850850 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.879647 (+ 28797us) server_negotiation.cc:407] Connection header received
0810 14:38:26.879943 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.879949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.880042 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.880152 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:26.882894 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.884080 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.886120 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.886823 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.886923 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.890715 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:26.890722 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:26.890727 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:26.890767 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:26.894627 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.895697 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.895703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.895707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.895892 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.896555 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.896560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.896563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.896926 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:26.897182 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.897880 (+ 698us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.898131 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:38:26.916695 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:26.880430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59428 (local address 127.21.55.124:35125)
0810 14:38:26.880611 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:26.880617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:26.899608 (+ 18991us) server_negotiation.cc:407] Connection header received
0810 14:38:26.899940 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:26.899944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:26.900021 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:26.900101 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:26.902105 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.903297 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.905146 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:26.905774 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:26.905902 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:26.909419 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:26.909426 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:26.909433 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:26.909473 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:26.913445 (+ 3972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.914307 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.914311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.914314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.914456 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:26.914959 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:26.914966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:26.914970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:26.915317 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:26.915604 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:26.915946 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 14:38:26.916236 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:38:27.023347 31753 catalog_manager.cc:4841] TS 4cd99a5acdd841648f05b96fd1a67819 (127.21.55.67:41079): tablet ba1fb063ae1e4567bdf66446ad27e08b (table test-table [id=423e27bdf6044a66857f5c9735ab30ca]) successfully deleted
I20260810 14:38:27.024113 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 31721
I20260810 14:38:27.084589 1492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:27.065925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44150 (local address 127.21.55.125:35855)
0810 14:38:27.066311 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:27.066321 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:27.067416 (+ 1095us) server_negotiation.cc:407] Connection header received
0810 14:38:27.067730 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:27.067741 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:27.067852 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:27.067948 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:27.070787 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:27.071746 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:27.073332 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:27.074104 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:27.074251 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:27.077708 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:27.077714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:27.077719 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:27.077760 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:27.081430 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:27.082279 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:27.082291 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:27.082294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:27.082410 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:27.082867 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:27.082872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:27.082874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:27.083164 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:27.083417 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:27.083795 (+ 378us) server_negotiation.cc:299] Negotiation successful
0810 14:38:27.084191 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260810 14:38:27.086359 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:27.067446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59438 (local address 127.21.55.124:35125)
0810 14:38:27.067686 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:27.067692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:27.069278 (+ 1586us) server_negotiation.cc:407] Connection header received
0810 14:38:27.070063 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:27.070070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:27.070154 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:27.070269 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:27.072896 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:27.073778 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:27.075553 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:27.076246 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:27.076394 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:27.079572 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:27.079579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:27.079584 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:27.079648 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:27.083468 (+ 3820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:27.084258 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:27.084263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:27.084267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:27.084399 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:27.084812 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:27.084817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:27.084820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:27.085115 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:27.085292 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:27.085774 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 14:38:27.086013 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:38:27.830221 1598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:27.829303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.66:60913)
0810 14:38:27.829845 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:27.829979 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":290,"threads_started":1}
W20260810 14:38:27.830713 1241 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:45213 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
I20260810 14:38:27.833702 1599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:27.832746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.67:41907)
0810 14:38:27.833271 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:27.833462 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":283,"threads_started":1}
W20260810 14:38:27.834159 1488 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:45213 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
I20260810 14:38:27.836047 1600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:27.835420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.65:40017)
0810 14:38:27.835800 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:27.835903 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":203,"threads_started":1}
W20260810 14:38:27.836418 995 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:45213 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
I20260810 14:38:28.346899 1603 raft_consensus.cc:482] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b56119bd614ca08ea2f06f6b4b9f40)
I20260810 14:38:28.347417 1603 raft_consensus.cc:504] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } }
I20260810 14:38:28.347898 1604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d1b56119bd614ca08ea2f06f6b4b9f40)
I20260810 14:38:28.348219 1604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } }
I20260810 14:38:28.349682 1604 leader_election.cc:288] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1b56119bd614ca08ea2f06f6b4b9f40 (127.21.55.126:45213), bae9e502167c42dda288e1d87c3eed23 (127.21.55.124:35125)
I20260810 14:38:28.349835 1603 leader_election.cc:288] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d1b56119bd614ca08ea2f06f6b4b9f40 (127.21.55.126:45213), 6e037c6f6e184e72bd3988d9bd7ca6e3 (127.21.55.125:35855)
I20260810 14:38:28.351394 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.350119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:54460)
0810 14:38:28.350828 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.351054 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":373,"threads_started":1}
W20260810 14:38:28.358717 324 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
I20260810 14:38:28.365336 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.350158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:54470)
0810 14:38:28.364543 (+ 14385us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.364934 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14130,"threads_started":1}
W20260810 14:38:28.366078 655 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
I20260810 14:38:28.368979 1613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.368600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:54490)
0810 14:38:28.368755 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.368839 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 14:38:28.369307 655 leader_election.cc:334] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b56119bd614ca08ea2f06f6b4b9f40 (127.21.55.126:45213): Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
I20260810 14:38:28.369632 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.351967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:35125 (local address 127.0.0.1:59444)
0810 14:38:28.352549 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.352575 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:28.352809 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:28.358174 (+ 5365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:28.358179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:28.358217 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:28.358584 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.358598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.360946 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.360956 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.364907 (+ 3951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.364944 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.367951 (+ 3007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.367959 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.368906 (+ 947us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.368950 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 14:38:28.369086 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 14:38:28.369280 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429}
I20260810 14:38:28.370163 1614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.351268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59444 (local address 127.21.55.124:35125)
0810 14:38:28.357592 (+ 6324us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:28.357600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:28.357626 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:38:28.357754 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:28.357760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:28.357995 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:28.358145 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:28.358831 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.360690 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.365507 (+ 4817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.367736 (+ 2229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.367883 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.367977 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:28.369706 (+ 1729us) server_negotiation.cc:299] Negotiation successful
0810 14:38:28.369843 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6152,"thread_start_us":5670,"threads_started":1}
I20260810 14:38:28.370205 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.367164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:54478)
0810 14:38:28.369853 (+ 2689us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.370005 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2586,"threads_started":1}
W20260810 14:38:28.370759 324 leader_election.cc:334] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d1b56119bd614ca08ea2f06f6b4b9f40 (127.21.55.126:45213): Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
I20260810 14:38:28.371745 700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bae9e502167c42dda288e1d87c3eed23" is_pre_election: true
I20260810 14:38:28.372197 700 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e037c6f6e184e72bd3988d9bd7ca6e3 in term 1.
I20260810 14:38:28.372783 324 leader_election.cc:302] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e037c6f6e184e72bd3988d9bd7ca6e3, bae9e502167c42dda288e1d87c3eed23; no voters: d1b56119bd614ca08ea2f06f6b4b9f40
I20260810 14:38:28.373123 1604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 14:38:28.373327 1604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d1b56119bd614ca08ea2f06f6b4b9f40)
I20260810 14:38:28.373495 1604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:38:28.375631 1612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.350334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:35855 (local address 127.0.0.1:44164)
0810 14:38:28.365338 (+ 15004us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.365361 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:28.365557 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:28.366744 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:28.366749 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:28.366785 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:28.367328 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.367341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.370985 (+ 3644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.370993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.372909 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.372930 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.374732 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.374739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.374946 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.374972 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 14:38:28.375086 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 14:38:28.375230 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14869,"thread_start_us":15188,"threads_started":1}
I20260810 14:38:28.375741 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.352196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.21.55.125:35855)
0810 14:38:28.357592 (+ 5396us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:28.357600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:28.365475 (+ 7875us) server_negotiation.cc:407] Connection header received
0810 14:38:28.365745 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:28.365754 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:28.365911 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:28.366075 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:28.367563 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.370284 (+ 2721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.373125 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.374486 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.374687 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.374844 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:28.375138 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 14:38:28.375303 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5232,"thread_start_us":6099,"threads_started":1}
I20260810 14:38:28.377061 364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bae9e502167c42dda288e1d87c3eed23" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" is_pre_election: true
I20260810 14:38:28.377061 1604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } }
I20260810 14:38:28.377786 364 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bae9e502167c42dda288e1d87c3eed23 in current term 2: Already voted for candidate 6e037c6f6e184e72bd3988d9bd7ca6e3 in this term.
I20260810 14:38:28.378135 1604 leader_election.cc:288] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [CANDIDATE]: Term 2 election: Requested vote from peers d1b56119bd614ca08ea2f06f6b4b9f40 (127.21.55.126:45213), bae9e502167c42dda288e1d87c3eed23 (127.21.55.124:35125)
I20260810 14:38:28.379458 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.378484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:54494)
0810 14:38:28.378816 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.379179 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260810 14:38:28.379382 658 leader_election.cc:302] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bae9e502167c42dda288e1d87c3eed23; no voters: 6e037c6f6e184e72bd3988d9bd7ca6e3, d1b56119bd614ca08ea2f06f6b4b9f40
I20260810 14:38:28.379412 700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bae9e502167c42dda288e1d87c3eed23"
I20260810 14:38:28.379704 700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:38:28.380820 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.380412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:54502)
0810 14:38:28.380591 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.380694 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 14:38:28.381043 324 leader_election.cc:334] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d1b56119bd614ca08ea2f06f6b4b9f40 (127.21.55.126:45213): Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
I20260810 14:38:28.382853 700 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e037c6f6e184e72bd3988d9bd7ca6e3 in term 2.
I20260810 14:38:28.383175 1603 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:38:28.384240 324 leader_election.cc:302] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6e037c6f6e184e72bd3988d9bd7ca6e3, bae9e502167c42dda288e1d87c3eed23; no voters: d1b56119bd614ca08ea2f06f6b4b9f40
I20260810 14:38:28.384752 1604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:38:28.385389 1604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [term 2 LEADER]: Becoming Leader. State: Replica: 6e037c6f6e184e72bd3988d9bd7ca6e3, State: Running, Role: LEADER
I20260810 14:38:28.385890 1604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } }
I20260810 14:38:28.388618 1629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e037c6f6e184e72bd3988d9bd7ca6e3. Latest consensus state: current_term: 2 leader_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:28.389139 1629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:28.389986 1631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:28.395653 1631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:28.397243 1631 catalog_manager.cc:1259] Loaded cluster ID: e7c59bf2e6034f37b02a806d9adc7a34
I20260810 14:38:28.397478 1631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:28.399202 1631 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:28.400158 1631 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: Loaded TSK: 0
I20260810 14:38:28.401652 1631 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:38:28.444621 348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:38:28.453647 348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:38:28.459457 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.458784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:54532)
0810 14:38:28.459027 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.459170 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 14:38:28.459995 700 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [term 2 FOLLOWER]: Refusing update from remote peer 6e037c6f6e184e72bd3988d9bd7ca6e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 14:38:28.460721 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.460316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:54548)
0810 14:38:28.460489 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.460562 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 14:38:28.461220 1629 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 14:38:28.461598 324 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 -> Peer d1b56119bd614ca08ea2f06f6b4b9f40 (127.21.55.126:45213): Couldn't send request to peer d1b56119bd614ca08ea2f06f6b4b9f40. Status: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 14:38:28.464866 1603 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e037c6f6e184e72bd3988d9bd7ca6e3. Latest consensus state: current_term: 2 leader_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:28.465233 1603 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:38:28.467087 1604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:28.467545 1604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:28.468394 1603 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d1b56119bd614ca08ea2f06f6b4b9f40" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45213 } } peers { permanent_uuid: "6e037c6f6e184e72bd3988d9bd7ca6e3" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 35855 } attrs { promote: false } } peers { permanent_uuid: "bae9e502167c42dda288e1d87c3eed23" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 35125 } attrs { promote: false } } }
I20260810 14:38:28.468732 1603 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae9e502167c42dda288e1d87c3eed23 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:38:28.468443 1633 mvcc.cc:205] Tried to move back new op lower bound from 7316982613841547264 to 7316982613553184768. Current Snapshot: MvccSnapshot[applied={T|T < 7316982613841547264}]
I20260810 14:38:28.501368 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.478728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59078 (local address 127.21.55.66:40743)
0810 14:38:28.491436 (+ 12708us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:28.491444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:28.491473 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:38:28.491612 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:28.491619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:28.491744 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:28.493085 (+ 1341us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:28.493315 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.494790 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.498167 (+ 3377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.499749 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.499964 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.500126 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:28.500750 (+ 624us) server_negotiation.cc:299] Negotiation successful
0810 14:38:28.500948 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12528,"thread_start_us":13206,"threads_started":1}
I20260810 14:38:28.501513 1622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.478279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:40743 (local address 127.0.0.1:59078)
0810 14:38:28.478501 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.478525 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:28.478687 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:28.492316 (+ 13629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:28.492323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:28.492362 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:28.492887 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.492906 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.494989 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.494996 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.497930 (+ 2934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.497941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.500136 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.500144 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.500375 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.500452 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 14:38:28.500716 (+ 264us) client_negotiation.cc:240] Negotiation successful
0810 14:38:28.500915 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":69}
I20260810 14:38:28.503132 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.478761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:41079 (local address 127.0.0.1:42556)
0810 14:38:28.479069 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.479087 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:28.479191 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:28.494068 (+ 14877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:28.494074 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:28.494124 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:28.494436 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.494447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.497791 (+ 3344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.497798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.499967 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.499987 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.502144 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.502152 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.502476 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.502519 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 14:38:28.502681 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 14:38:28.502862 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209}
I20260810 14:38:28.503230 1126 tablet_service.cc:1467] Processing CreateTablet for tablet b39d8dce93004c089a64d1ec9493b00c (DEFAULT_TABLE table=test-table [id=33b4e1d96b0c4cf59ad10303b94e6ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:28.503672 1647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.479048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42556 (local address 127.21.55.67:41079)
0810 14:38:28.493705 (+ 14657us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:28.493711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:28.493730 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:38:28.493801 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:28.493805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:28.493885 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:28.494036 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:28.494620 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.497623 (+ 3003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.500266 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.501797 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.502093 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.502354 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:28.502954 (+ 600us) server_negotiation.cc:299] Negotiation successful
0810 14:38:28.503126 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14530,"thread_start_us":14401,"threads_started":1}
I20260810 14:38:28.503947 1126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b39d8dce93004c089a64d1ec9493b00c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:28.504755 1366 tablet_service.cc:1467] Processing CreateTablet for tablet b39d8dce93004c089a64d1ec9493b00c (DEFAULT_TABLE table=test-table [id=33b4e1d96b0c4cf59ad10303b94e6ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:28.505425 1366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b39d8dce93004c089a64d1ec9493b00c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:28.507906 1648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.479630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:34835 (local address 127.0.0.1:47864)
0810 14:38:28.491804 (+ 12174us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.491877 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:28.492065 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:28.493024 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:28.493033 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:28.493078 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:28.493507 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.493520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.495292 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.495298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.497889 (+ 2591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.497915 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.506854 (+ 8939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.506862 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.507073 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.507118 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 14:38:28.507244 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 14:38:28.507412 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11994,"mutex_wait_us":42,"thread_start_us":11783,"threads_started":1}
I20260810 14:38:28.508324 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.479856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.21.55.65:34835)
0810 14:38:28.491001 (+ 11145us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:28.491012 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:28.492385 (+ 1373us) server_negotiation.cc:407] Connection header received
0810 14:38:28.492520 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:28.492529 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:28.492646 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:28.492785 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:28.493728 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.495120 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.498808 (+ 3688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.499995 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.500193 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.500345 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:28.507368 (+ 7023us) server_negotiation.cc:299] Negotiation successful
0810 14:38:28.507611 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10887,"thread_start_us":11182,"threads_started":1}
I20260810 14:38:28.509439 887 tablet_service.cc:1467] Processing CreateTablet for tablet b39d8dce93004c089a64d1ec9493b00c (DEFAULT_TABLE table=test-table [id=33b4e1d96b0c4cf59ad10303b94e6ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:28.509963 887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b39d8dce93004c089a64d1ec9493b00c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:28.511967 1652 tablet_bootstrap.cc:492] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819: Bootstrap starting.
I20260810 14:38:28.513583 1652 tablet_bootstrap.cc:654] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:28.517532 1655 tablet_bootstrap.cc:492] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792: Bootstrap starting.
I20260810 14:38:28.518002 1654 tablet_bootstrap.cc:492] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf: Bootstrap starting.
I20260810 14:38:28.518847 1652 tablet_bootstrap.cc:492] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819: No bootstrap required, opened a new log
I20260810 14:38:28.519052 1652 ts_tablet_manager.cc:1397] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260810 14:38:28.520253 1655 tablet_bootstrap.cc:654] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:28.520052 1652 raft_consensus.cc:348] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:28.520362 1652 raft_consensus.cc:374] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:28.520486 1652 raft_consensus.cc:729] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd99a5acdd841648f05b96fd1a67819, State: Initialized, Role: FOLLOWER
I20260810 14:38:28.520751 1652 consensus_queue.cc:260] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:28.521157 1654 tablet_bootstrap.cc:654] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:28.529701 1652 ts_tablet_manager.cc:1428] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819: Time spent starting tablet: real 0.010s user 0.005s sys 0.003s
I20260810 14:38:28.531661 1655 tablet_bootstrap.cc:492] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792: No bootstrap required, opened a new log
I20260810 14:38:28.533548 1655 ts_tablet_manager.cc:1397] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260810 14:38:28.534916 1655 raft_consensus.cc:348] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:28.535308 1655 raft_consensus.cc:374] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:28.535476 1655 raft_consensus.cc:729] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6954977858545a6b1890c543c44b792, State: Initialized, Role: FOLLOWER
I20260810 14:38:28.535894 1655 consensus_queue.cc:260] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:28.538565 1489 heartbeater.cc:503] Master 127.21.55.125:35855 was elected leader, sending a full tablet report...
I20260810 14:38:28.550565 1655 ts_tablet_manager.cc:1428] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792: Time spent starting tablet: real 0.017s user 0.007s sys 0.007s
I20260810 14:38:28.552955 1654 tablet_bootstrap.cc:492] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf: No bootstrap required, opened a new log
I20260810 14:38:28.553176 1654 ts_tablet_manager.cc:1397] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf: Time spent bootstrapping tablet: real 0.035s user 0.008s sys 0.024s
I20260810 14:38:28.553241 1242 heartbeater.cc:503] Master 127.21.55.125:35855 was elected leader, sending a full tablet report...
I20260810 14:38:28.554306 1654 raft_consensus.cc:348] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:28.554637 1654 raft_consensus.cc:374] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:28.554769 1654 raft_consensus.cc:729] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d362b4225743e481b4a4f989a11ebf, State: Initialized, Role: FOLLOWER
I20260810 14:38:28.555047 1654 consensus_queue.cc:260] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:28.558633 1663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.551176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.66:37151)
0810 14:38:28.557768 (+ 6592us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.558108 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":4497,"threads_started":1}
I20260810 14:38:28.561456 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.532312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.67:52019)
0810 14:38:28.560909 (+ 28597us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.561105 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28451,"thread_start_us":28293,"threads_started":1}
I20260810 14:38:28.561937 1663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.560909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.66:41469)
0810 14:38:28.561575 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.561686 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427}
I20260810 14:38:28.562422 1654 ts_tablet_manager.cc:1428] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260810 14:38:28.564754 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.564098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.67:45369)
0810 14:38:28.564267 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.564362 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 14:38:28.565526 1665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.563252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.65:38255)
0810 14:38:28.564937 (+ 1685us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.565163 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1501,"thread_start_us":1345,"threads_started":1}
W20260810 14:38:28.624363 997 tablet.cc:2369] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:38:28.831117 1661 raft_consensus.cc:482] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:38:28.831375 1661 raft_consensus.cc:504] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:28.833204 1661 leader_election.cc:288] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cd99a5acdd841648f05b96fd1a67819 (127.21.55.67:41079), b4d362b4225743e481b4a4f989a11ebf (127.21.55.65:34835)
I20260810 14:38:28.839557 1647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.833768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:54185 (local address 127.21.55.67:41079)
0810 14:38:28.834038 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:28.834049 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:28.834082 (+ 33us) server_negotiation.cc:407] Connection header received
0810 14:38:28.834284 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:28.834290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:28.834356 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:28.834461 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:28.835108 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.835827 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.837703 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.838502 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.838750 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.838832 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:28.839232 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 14:38:28.839336 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:38:28.840178 1663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.833635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:41079 (local address 127.21.55.66:54185)
0810 14:38:28.833919 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.833937 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:28.834085 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:28.834497 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:28.834501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:28.834552 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:28.834869 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.834879 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.835969 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.835976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.837528 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.837543 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.838777 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.838788 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.838972 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.839052 (+ 80us) client_negotiation.cc:769] Sending connection context
0810 14:38:28.839764 (+ 712us) client_negotiation.cc:240] Negotiation successful
0810 14:38:28.839924 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260810 14:38:28.841148 1406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39d8dce93004c089a64d1ec9493b00c" candidate_uuid: "f6954977858545a6b1890c543c44b792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd99a5acdd841648f05b96fd1a67819" is_pre_election: true
I20260810 14:38:28.841610 1406 raft_consensus.cc:2454] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6954977858545a6b1890c543c44b792 in term 0.
I20260810 14:38:28.841717 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.834399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:52279 (local address 127.21.55.65:34835)
0810 14:38:28.834723 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:28.834732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:28.834951 (+ 219us) server_negotiation.cc:407] Connection header received
0810 14:38:28.835248 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:28.835253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:28.835327 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:28.835449 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:28.836091 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.836973 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.839055 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.840355 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.840597 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.840684 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:28.841277 (+ 593us) server_negotiation.cc:299] Negotiation successful
0810 14:38:28.841430 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 14:38:28.841884 1666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.833798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:34835 (local address 127.21.55.66:52279)
0810 14:38:28.834766 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.834794 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:28.835027 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:28.835473 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:28.835478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:28.835505 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:28.835900 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.835910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.837137 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.837146 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.838864 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:28.838879 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.840878 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.840888 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:28.841065 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.841092 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 14:38:28.841221 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 14:38:28.841376 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":941,"threads_started":1}
I20260810 14:38:28.842278 1056 leader_election.cc:302] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd99a5acdd841648f05b96fd1a67819, f6954977858545a6b1890c543c44b792; no voters:
I20260810 14:38:28.842705 1661 raft_consensus.cc:2781] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:38:28.842660 920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39d8dce93004c089a64d1ec9493b00c" candidate_uuid: "f6954977858545a6b1890c543c44b792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d362b4225743e481b4a4f989a11ebf" is_pre_election: true
I20260810 14:38:28.842831 1661 raft_consensus.cc:482] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:38:28.842902 1661 raft_consensus.cc:3037] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:28.843003 920 raft_consensus.cc:2454] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6954977858545a6b1890c543c44b792 in term 0.
I20260810 14:38:28.846149 1661 raft_consensus.cc:504] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:28.846974 1661 leader_election.cc:288] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [CANDIDATE]: Term 1 election: Requested vote from peers 4cd99a5acdd841648f05b96fd1a67819 (127.21.55.67:41079), b4d362b4225743e481b4a4f989a11ebf (127.21.55.65:34835)
I20260810 14:38:28.847720 1406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39d8dce93004c089a64d1ec9493b00c" candidate_uuid: "f6954977858545a6b1890c543c44b792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd99a5acdd841648f05b96fd1a67819"
I20260810 14:38:28.847966 1406 raft_consensus.cc:3037] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:28.847796 920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b39d8dce93004c089a64d1ec9493b00c" candidate_uuid: "f6954977858545a6b1890c543c44b792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d362b4225743e481b4a4f989a11ebf"
I20260810 14:38:28.848409 920 raft_consensus.cc:3037] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:28.851270 1406 raft_consensus.cc:2454] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6954977858545a6b1890c543c44b792 in term 1.
I20260810 14:38:28.851908 1056 leader_election.cc:302] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd99a5acdd841648f05b96fd1a67819, f6954977858545a6b1890c543c44b792; no voters:
I20260810 14:38:28.852430 1661 raft_consensus.cc:2781] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:28.852396 920 raft_consensus.cc:2454] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6954977858545a6b1890c543c44b792 in term 1.
I20260810 14:38:28.853703 1661 raft_consensus.cc:686] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 1 LEADER]: Becoming Leader. State: Replica: f6954977858545a6b1890c543c44b792, State: Running, Role: LEADER
I20260810 14:38:28.854339 1661 consensus_queue.cc:237] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } }
I20260810 14:38:28.855814 1666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.855020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.66:46609)
0810 14:38:28.855341 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.855520 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 14:38:28.859803 349 catalog_manager.cc:5495] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6954977858545a6b1890c543c44b792 (127.21.55.66). New cstate: current_term: 1 leader_uuid: "f6954977858545a6b1890c543c44b792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6954977858545a6b1890c543c44b792" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260810 14:38:28.971418 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.964763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59094 (local address 127.21.55.66:40743)
0810 14:38:28.965181 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:28.965189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:28.965508 (+ 319us) server_negotiation.cc:407] Connection header received
0810 14:38:28.965830 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:28.965835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:28.965940 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:28.966127 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:38:28.966600 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.967388 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.968919 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:28.969398 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:28.969553 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:28.970024 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:38:28.970707 (+ 683us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:38:28.970875 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:28.971000 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 14:38:28.971103 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260810 14:38:28.977566 920 raft_consensus.cc:1261] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf [term 1 FOLLOWER]: Refusing update from remote peer f6954977858545a6b1890c543c44b792: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:28.977835 1406 raft_consensus.cc:1261] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819 [term 1 FOLLOWER]: Refusing update from remote peer f6954977858545a6b1890c543c44b792: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:28.978690 1667 consensus_queue.cc:1035] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d362b4225743e481b4a4f989a11ebf" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:28.979275 1661 consensus_queue.cc:1035] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd99a5acdd841648f05b96fd1a67819" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:28.981545 996 heartbeater.cc:503] Master 127.21.55.125:35855 was elected leader, sending a full tablet report...
I20260810 14:38:28.983084 1665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.982378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.65:54145)
0810 14:38:28.982734 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.982856 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
I20260810 14:38:28.983631 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.983117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.67:40575)
0810 14:38:28.983302 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.983433 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 14:38:28.993064 1666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.992262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.66:41275)
0810 14:38:28.992570 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.992722 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260810 14:38:28.993556 1684 mvcc.cc:205] Tried to move back new op lower bound from 7316982615960006656 to 7316982615472734208. Current Snapshot: MvccSnapshot[applied={T|T < 7316982615960006656}]
I20260810 14:38:28.994624 1665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.993776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.65:32961)
0810 14:38:28.994029 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.994333 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260810 14:38:28.995985 1659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:28.995185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.21.55.67:52591)
0810 14:38:28.995570 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:28.995703 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220}
I20260810 14:38:29.001735 1683 mvcc.cc:205] Tried to move back new op lower bound from 7316982615960006656 to 7316982615472734208. Current Snapshot: MvccSnapshot[applied={T|T < 7316982615960006656}]
I20260810 14:38:29.014727 349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:29.015004 349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:29.022482 349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e037c6f6e184e72bd3988d9bd7ca6e3: Sending DeleteTablet for 3 replicas of tablet b39d8dce93004c089a64d1ec9493b00c
I20260810 14:38:29.024405 1366 tablet_service.cc:1514] Processing DeleteTablet for tablet b39d8dce93004c089a64d1ec9493b00c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:29 UTC) from {username='oryx', principal='oryx/127.21.55.125@KRBTEST.COM'} at 127.0.0.1:42556
I20260810 14:38:29.024400 887 tablet_service.cc:1514] Processing DeleteTablet for tablet b39d8dce93004c089a64d1ec9493b00c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:29 UTC) from {username='oryx', principal='oryx/127.21.55.125@KRBTEST.COM'} at 127.0.0.1:47864
I20260810 14:38:29.024967 1126 tablet_service.cc:1514] Processing DeleteTablet for tablet b39d8dce93004c089a64d1ec9493b00c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:29 UTC) from {username='oryx', principal='oryx/127.21.55.125@KRBTEST.COM'} at 127.0.0.1:59078
I20260810 14:38:29.026002 1696 tablet_replica.cc:330] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792: stopping tablet replica
I20260810 14:38:29.027037 1696 raft_consensus.cc:2229] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:38:29.027987 1696 raft_consensus.cc:2258] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:29.029448 1695 tablet_replica.cc:330] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf: stopping tablet replica
I20260810 14:38:29.029445 1696 ts_tablet_manager.cc:1905] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:29.029706 1695 raft_consensus.cc:2229] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:29.030088 1695 raft_consensus.cc:2258] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:29.031497 1695 ts_tablet_manager.cc:1905] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:29.033146 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 636
I20260810 14:38:29.039153 1696 ts_tablet_manager.cc:1918] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:29.039153 1695 ts_tablet_manager.cc:1918] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:29.039326 1695 log.cc:1196] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/b39d8dce93004c089a64d1ec9493b00c
I20260810 14:38:29.039767 1696 log.cc:1196] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/b39d8dce93004c089a64d1ec9493b00c
I20260810 14:38:29.039880 1695 ts_tablet_manager.cc:1939] T b39d8dce93004c089a64d1ec9493b00c P b4d362b4225743e481b4a4f989a11ebf: Deleting consensus metadata
I20260810 14:38:29.040515 1696 ts_tablet_manager.cc:1939] T b39d8dce93004c089a64d1ec9493b00c P f6954977858545a6b1890c543c44b792: Deleting consensus metadata
I20260810 14:38:29.041443 324 catalog_manager.cc:4841] TS b4d362b4225743e481b4a4f989a11ebf (127.21.55.65:34835): tablet b39d8dce93004c089a64d1ec9493b00c (table test-table [id=33b4e1d96b0c4cf59ad10303b94e6ef0]) successfully deleted
I20260810 14:38:29.041973 319 catalog_manager.cc:4841] TS f6954977858545a6b1890c543c44b792 (127.21.55.66:40743): tablet b39d8dce93004c089a64d1ec9493b00c (table test-table [id=33b4e1d96b0c4cf59ad10303b94e6ef0]) successfully deleted
I20260810 14:38:29.043752 1694 tablet_replica.cc:330] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819: stopping tablet replica
I20260810 14:38:29.044370 1694 raft_consensus.cc:2229] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:29.044864 1694 raft_consensus.cc:2258] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:29.046295 1694 ts_tablet_manager.cc:1905] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:29.054679 1694 ts_tablet_manager.cc:1918] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:29.054904 1694 log.cc:1196] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/b39d8dce93004c089a64d1ec9493b00c
I20260810 14:38:29.055455 1694 ts_tablet_manager.cc:1939] T b39d8dce93004c089a64d1ec9493b00c P 4cd99a5acdd841648f05b96fd1a67819: Deleting consensus metadata
I20260810 14:38:29.056994 327 catalog_manager.cc:4841] TS 4cd99a5acdd841648f05b96fd1a67819 (127.21.55.67:41079): tablet b39d8dce93004c089a64d1ec9493b00c (table test-table [id=33b4e1d96b0c4cf59ad10303b94e6ef0]) successfully deleted
I20260810 14:38:29.060606 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:29.034394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44686 (local address 127.21.55.65:34835)
0810 14:38:29.034648 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:29.034656 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:29.041697 (+ 7041us) server_negotiation.cc:407] Connection header received
0810 14:38:29.042396 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:29.042401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:29.042498 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:29.042699 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:29.044671 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:29.045870 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:29.047378 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:29.047830 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:29.048002 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:29.051501 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:29.051521 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:29.051542 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:29.051584 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:29.056683 (+ 5099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:29.057832 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:29.057841 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:29.057843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:29.058031 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:29.058499 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:29.058505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:29.058509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:29.059081 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:29.059456 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:29.059824 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 14:38:29.060189 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 14:38:29.061480 982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:29.064976 1699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:29.063710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:56682)
0810 14:38:29.064378 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:29.064560 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":346,"threads_started":1}
I20260810 14:38:29.066975 1699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:29.066431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:56692)
0810 14:38:29.066683 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:29.066804 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 14:38:29.183954 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 636
I20260810 14:38:29.209348 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 1026
I20260810 14:38:29.227280 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:29.210362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.21.55.66:40743)
0810 14:38:29.210701 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:29.210709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:29.210738 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:38:29.210840 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:29.210848 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:29.210980 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:29.211115 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:29.212510 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:29.213864 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:29.215336 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:29.216068 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:29.216224 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:29.218961 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:29.218977 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:29.219009 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:29.219060 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:29.223838 (+ 4778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:29.224646 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:29.224664 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:29.224668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:29.224892 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:29.225327 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:29.225332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:29.225336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:29.225889 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:29.226308 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:29.226495 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 14:38:29.226837 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260810 14:38:29.228367 1227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:29.359042 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 1026
I20260810 14:38:29.385834 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 1264
I20260810 14:38:29.401422 1705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:29.386715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.21.55.67:41079)
0810 14:38:29.387368 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:29.387372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:29.387391 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:38:29.387449 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:29.387453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:29.387514 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:29.387635 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:29.389053 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:29.389775 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:29.391228 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:29.391630 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:29.391735 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:29.394203 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:29.394216 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:29.394234 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:29.394263 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:29.397489 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:29.398148 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:29.398155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:29.398157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:29.398312 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:29.399000 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:29.399007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:29.399010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:29.399753 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:29.400037 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:29.400691 (+ 654us) server_negotiation.cc:299] Negotiation successful
0810 14:38:29.400915 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":438,"threads_started":1}
I20260810 14:38:29.402146 1470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:29.523955 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 1264
I20260810 14:38:29.552374 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 32573
I20260810 14:38:29.553591 396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:29.694082 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:29.692931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45213 (local address 127.0.0.1:56700)
0810 14:38:29.693624 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:29.693779 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:45213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":416,"threads_started":1}
I20260810 14:38:29.694193 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 32573
W20260810 14:38:29.727912 659 connection.cc:535] server connection from 127.0.0.1:59444 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 14:38:29.728330 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 32753
I20260810 14:38:29.729696 736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:29.858335 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 32753
2026-08-10T14:38:29Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10948 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:38:29 dist-test-slave-h6n0 krb5kdc[1721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:38:29 dist-test-slave-h6n0 krb5kdc[1721](info): set up 2 sockets
Aug 10 14:38:29 dist-test-slave-h6n0 krb5kdc[1721](info): commencing operation
krb5kdc: starting...
W20260810 14:38:31.283133 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.365s user 0.007s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:38:31 dist-test-slave-h6n0 krb5kdc[1721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:38:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:38:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.126.keytab.
Entry for principal oryx/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.126.keytab.
I20260810 14:38:31.559669 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:43481
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.126.keytab
--principal=oryx/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:34963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:43481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:31.910833 1741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:31.911172 1741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:31.911332 1741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:31.918744 1741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:31.918890 1741 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 14:38:31.918964 1741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:31.919061 1741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:31.919139 1741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:31.919227 1741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:38:31.929774 1741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:43481
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.126.keytab
--principal=oryx/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:43481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:31.931156 1741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:31.932535 1741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:31.946590 1752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:31.946911 1748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:31.947197 1749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:31.948356 1741 server_base.cc:1034] running on GCE node
I20260810 14:38:31.949278 1741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:31.951041 1741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:31.952304 1741 hybrid_clock.cc:648] HybridClock initialized: now 1786372711952241 us; error 27 us; skew 500 ppm
I20260810 14:38:31.952684 1741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:31 dist-test-slave-h6n0 krb5kdc[1721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372711, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:31.959250 1741 init.cc:377] Logged in from keytab as oryx/127.21.55.126@KRBTEST.COM (short username oryx)
I20260810 14:38:31.960822 1741 webserver.cc:460] Webserver started at http://127.21.55.126:34839/ using document root <none> and password file <none>
I20260810 14:38:31.961550 1741 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:31.961689 1741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:31.962026 1741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:31.965077 1741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "08bd303162484c20bcfee8de64a443c6"
format_stamp: "Formatted at 2026-08-10 14:38:31 on dist-test-slave-h6n0"
server_key: "22cd92545a52bf572391971a86a359e7"
server_key_iv: "981fdcb11c9cd40787f1934eadc0ea72"
server_key_version: "encryptionkey@0"
I20260810 14:38:31.965883 1741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "08bd303162484c20bcfee8de64a443c6"
format_stamp: "Formatted at 2026-08-10 14:38:31 on dist-test-slave-h6n0"
server_key: "22cd92545a52bf572391971a86a359e7"
server_key_iv: "981fdcb11c9cd40787f1934eadc0ea72"
server_key_version: "encryptionkey@0"
I20260810 14:38:31.971000 1741 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:38:31.974455 1766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:31.976500 1741 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 14:38:31.976639 1741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "08bd303162484c20bcfee8de64a443c6"
format_stamp: "Formatted at 2026-08-10 14:38:31 on dist-test-slave-h6n0"
server_key: "22cd92545a52bf572391971a86a359e7"
server_key_iv: "981fdcb11c9cd40787f1934eadc0ea72"
server_key_version: "encryptionkey@0"
I20260810 14:38:31.976828 1741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:31.994503 1741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:31.999394 1741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:31.999696 1741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:32.025740 1741 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:43481
I20260810 14:38:32.025760 1871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:43481 every 8 connection(s)
I20260810 14:38:32.027172 1741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:38:32.031093 1873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:32.036509 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 1741
I20260810 14:38:32.036767 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:38:32.037467 21725 external_mini_cluster.cc:1428] Setting key 08e7b87e7078957d09bbbd30ac8973cd
I20260810 14:38:32.041872 1873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6: Bootstrap starting.
I20260810 14:38:32.046067 1873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:32.047197 1873 log.cc:824] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:32.050352 1873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6: No bootstrap required, opened a new log
Aug 10 14:38:32 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:32.060766 1873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08bd303162484c20bcfee8de64a443c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43481 } }
I20260810 14:38:32.061285 1873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:32.061399 1873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08bd303162484c20bcfee8de64a443c6, State: Initialized, Role: FOLLOWER
I20260810 14:38:32.062055 1873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08bd303162484c20bcfee8de64a443c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43481 } }
I20260810 14:38:32.062351 1873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:38:32.062506 1873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:38:32.062721 1873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:32.063162 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:32.039513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47046 (local address 127.21.55.126:43481)
0810 14:38:32.040344 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:32.040355 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:32.040404 (+ 49us) server_negotiation.cc:407] Connection header received
0810 14:38:32.042087 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:32.042114 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:32.042474 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:32.042831 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:32.045235 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:32.047098 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:32.048858 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:32.049625 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:32.049775 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:32.054408 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:32.054418 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:32.054435 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:32.054477 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:32.058641 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:32.059289 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:32.059298 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:32.059306 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:32.059517 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:32.059940 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:32.059946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:32.059950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:32.060651 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:32.060966 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:32.061327 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 14:38:32.061759 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":404,"threads_started":1}
I20260810 14:38:32.066380 1873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08bd303162484c20bcfee8de64a443c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43481 } }
I20260810 14:38:32.067139 1873 leader_election.cc:302] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08bd303162484c20bcfee8de64a443c6; no voters:
I20260810 14:38:32.068094 1873 leader_election.cc:288] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:38:32.068240 1885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:32.069162 1885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [term 1 LEADER]: Becoming Leader. State: Replica: 08bd303162484c20bcfee8de64a443c6, State: Running, Role: LEADER
I20260810 14:38:32.069795 1885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08bd303162484c20bcfee8de64a443c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43481 } }
I20260810 14:38:32.070508 1873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:32.072901 1888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08bd303162484c20bcfee8de64a443c6. Latest consensus state: current_term: 1 leader_uuid: "08bd303162484c20bcfee8de64a443c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08bd303162484c20bcfee8de64a443c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43481 } } }
I20260810 14:38:32.073266 1888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:32.074425 1886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08bd303162484c20bcfee8de64a443c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08bd303162484c20bcfee8de64a443c6" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43481 } } }
I20260810 14:38:32.074906 1886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:32.076627 1893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:32.082408 1893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:32.091131 1893 catalog_manager.cc:1347] Generated new cluster ID: 1bae8e2640d04a98bfcf5ba05d190485
I20260810 14:38:32.091250 1893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:32.101658 1893 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:38:32.102999 1893 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:32.110610 1893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6: Generated new TSK 0
I20260810 14:38:32.111403 1893 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab.
Entry for principal oryx/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab.
I20260810 14:38:32.178577 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43481
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab
--principal=oryx/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:34963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:32.544173 1922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:32.544472 1922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:32.544654 1922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:32.553339 1922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:32.553462 1922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:32.553661 1922 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:38:32.553802 1922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:38:32.564747 1922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab
--principal=oryx/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.55.126:43481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:32.566052 1922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:32.567408 1922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:32.583431 1934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:32.583158 1935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:32.583135 1938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:32.585227 1922 server_base.cc:1034] running on GCE node
I20260810 14:38:32.585978 1922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:32.587025 1922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:32.588279 1922 hybrid_clock.cc:648] HybridClock initialized: now 1786372712588214 us; error 43 us; skew 500 ppm
I20260810 14:38:32.588595 1922 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:32 dist-test-slave-h6n0 krb5kdc[1721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372712, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:32.592306 1922 init.cc:377] Logged in from keytab as oryx/127.21.55.65@KRBTEST.COM (short username oryx)
I20260810 14:38:32.593833 1922 webserver.cc:460] Webserver started at http://127.21.55.65:33869/ using document root <none> and password file <none>
I20260810 14:38:32.594638 1922 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:32.594779 1922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:32.595132 1922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:32.598080 1922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "04da3915c6bf4d5f9cd132f484029a80"
format_stamp: "Formatted at 2026-08-10 14:38:32 on dist-test-slave-h6n0"
server_key: "18da3aa505d800f56168cce88f6f6f4e"
server_key_iv: "bc92d62b3ec6b39c8044e89c4a86b2ca"
server_key_version: "encryptionkey@0"
I20260810 14:38:32.599006 1922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "04da3915c6bf4d5f9cd132f484029a80"
format_stamp: "Formatted at 2026-08-10 14:38:32 on dist-test-slave-h6n0"
server_key: "18da3aa505d800f56168cce88f6f6f4e"
server_key_iv: "bc92d62b3ec6b39c8044e89c4a86b2ca"
server_key_version: "encryptionkey@0"
I20260810 14:38:32.606671 1922 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 14:38:32.610798 1948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:32.612175 1922 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:38:32.612377 1922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "04da3915c6bf4d5f9cd132f484029a80"
format_stamp: "Formatted at 2026-08-10 14:38:32 on dist-test-slave-h6n0"
server_key: "18da3aa505d800f56168cce88f6f6f4e"
server_key_iv: "bc92d62b3ec6b39c8044e89c4a86b2ca"
server_key_version: "encryptionkey@0"
I20260810 14:38:32.612665 1922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:32.638801 1922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:32.643503 1922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:32.643801 1922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:32.646778 1922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:32.646885 1922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:32.647043 1922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:32.647128 1922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:38:32 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372712, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:32.672580 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:32.647438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:59737 (local address 127.21.55.126:43481)
0810 14:38:32.649881 (+ 2443us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:32.649888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:32.649911 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:38:32.651860 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:32.651868 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:32.651980 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:32.652141 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:32.654356 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:32.655719 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:32.657857 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:32.658346 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:32.658493 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:32.663734 (+ 5241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:32.663740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:32.663745 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:32.663791 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:32.668145 (+ 4354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:32.669194 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:32.669214 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:32.669221 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:32.669440 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:32.669962 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:32.669970 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:32.669976 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:32.670467 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:32.670695 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:32.671943 (+ 1248us) server_negotiation.cc:299] Negotiation successful
0810 14:38:32.672197 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2303,"thread_start_us":2171,"threads_started":1}
I20260810 14:38:32.673589 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:32.648077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43481 (local address 127.21.55.65:59737)
0810 14:38:32.649174 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:32.649249 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:32.651583 (+ 2334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:32.652405 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:32.652418 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:32.653232 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:32.654123 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:32.654152 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:32.655913 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:32.655917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:32.657696 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:32.657709 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:32.658647 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:32.658656 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:32.658906 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:32.659831 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:32.659874 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:32.663409 (+ 3535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:32.668318 (+ 4909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:32.668332 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:32.668344 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:32.668842 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:32.669608 (+ 766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:32.669612 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:32.669614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:32.669817 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:32.670655 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:32.670661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:32.671502 (+ 841us) client_negotiation.cc:769] Sending connection context
0810 14:38:32.671826 (+ 324us) client_negotiation.cc:240] Negotiation successful
0810 14:38:32.672360 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":452,"threads_started":1}
I20260810 14:38:32.697971 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:32.681480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:54943 (local address 127.21.55.126:43481)
0810 14:38:32.681711 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:32.681718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:32.682675 (+ 957us) server_negotiation.cc:407] Connection header received
0810 14:38:32.682868 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:32.682880 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:32.683011 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:32.683142 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:32.685874 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:32.686848 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:32.688962 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:32.689572 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:32.689711 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:32.692198 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:32.692203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:32.692206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:32.692268 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:32.695307 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:32.695991 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:32.695997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:32.696000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:32.696133 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:32.696509 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:32.696514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:32.696517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:32.696783 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:32.696954 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:32.697373 (+ 419us) server_negotiation.cc:299] Negotiation successful
0810 14:38:32.697605 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:38:32.698082 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:32.681432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43481 (local address 127.21.55.65:54943)
0810 14:38:32.682411 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:32.682439 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:32.682651 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:32.683563 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:32.683570 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:32.684679 (+ 1109us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:32.685613 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:32.685637 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:32.686999 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:32.687005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:32.688808 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:32.688821 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:32.689781 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:32.689789 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:32.689954 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:32.690650 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:32.690667 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:32.691893 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:32.695453 (+ 3560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:32.695458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:32.695463 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:32.695862 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:32.696249 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:32.696254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:32.696258 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:32.696394 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:32.696908 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:32.696913 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:32.697101 (+ 188us) client_negotiation.cc:769] Sending connection context
0810 14:38:32.697264 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 14:38:32.697606 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":462,"threads_started":1}
I20260810 14:38:32.708705 1922 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:40573
I20260810 14:38:32.708741 2151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:40573 every 8 connection(s)
I20260810 14:38:32.709939 1922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:38:32.712750 2152 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43481
I20260810 14:38:32.712883 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 1922
I20260810 14:38:32.713119 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:38:32.713280 2152 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:32.713786 21725 external_mini_cluster.cc:1428] Setting key 32f0108f2ff22adf4b42e6c2a5454564
I20260810 14:38:32.714361 2152 heartbeater.cc:511] Master 127.21.55.126:43481 requested a full tablet report, sending...
I20260810 14:38:32.716804 1796 ts_manager.cc:194] Registered new tserver with Master: 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573)
I20260810 14:38:32.718434 1796 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:59737
WARNING: no policy specified for oryx/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab.
Entry for principal oryx/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab.
I20260810 14:38:32.770467 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43481
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab
--principal=oryx/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:34963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:33.119104 2160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:33.119370 2160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:33.119472 2160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:33.126487 2160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:33.126648 2160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:33.126744 2160 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:38:33.126827 2160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:38:33.136399 2160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab
--principal=oryx/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.55.126:43481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:33.137600 2160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:33.138903 2160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:33.151939 2167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:33.152076 2168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:33.152614 2170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:33.152865 2160 server_base.cc:1034] running on GCE node
I20260810 14:38:33.153601 2160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:33.154621 2160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:33.155894 2160 hybrid_clock.cc:648] HybridClock initialized: now 1786372713155814 us; error 50 us; skew 500 ppm
I20260810 14:38:33.156224 2160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:33 dist-test-slave-h6n0 krb5kdc[1721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372713, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:33.161545 2160 init.cc:377] Logged in from keytab as oryx/127.21.55.66@KRBTEST.COM (short username oryx)
I20260810 14:38:33.163288 2160 webserver.cc:460] Webserver started at http://127.21.55.66:42349/ using document root <none> and password file <none>
I20260810 14:38:33.164067 2160 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:33.164227 2160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:33.164618 2160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:33.167954 2160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "894207e400bc4b91ad71973a424b3c49"
format_stamp: "Formatted at 2026-08-10 14:38:33 on dist-test-slave-h6n0"
server_key: "8d617aed2573cb6f6e8cab5bc139124e"
server_key_iv: "28467a2c2b7f7e6ceb2ddc8e5c5629d7"
server_key_version: "encryptionkey@0"
I20260810 14:38:33.168890 2160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "894207e400bc4b91ad71973a424b3c49"
format_stamp: "Formatted at 2026-08-10 14:38:33 on dist-test-slave-h6n0"
server_key: "8d617aed2573cb6f6e8cab5bc139124e"
server_key_iv: "28467a2c2b7f7e6ceb2ddc8e5c5629d7"
server_key_version: "encryptionkey@0"
I20260810 14:38:33.174892 2160 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260810 14:38:33.179600 2184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:33.181332 2160 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 14:38:33.181516 2160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "894207e400bc4b91ad71973a424b3c49"
format_stamp: "Formatted at 2026-08-10 14:38:33 on dist-test-slave-h6n0"
server_key: "8d617aed2573cb6f6e8cab5bc139124e"
server_key_iv: "28467a2c2b7f7e6ceb2ddc8e5c5629d7"
server_key_version: "encryptionkey@0"
I20260810 14:38:33.181754 2160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:33.200978 2160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:33.206925 2160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:33.207237 2160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:33.210436 2160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:33.210595 2160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:33.210793 2160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:33.210889 2160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:38:33 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372713, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:33.236040 2196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:33.212256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:47225 (local address 127.21.55.126:43481)
0810 14:38:33.212800 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:33.212809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:33.214107 (+ 1298us) server_negotiation.cc:407] Connection header received
0810 14:38:33.217016 (+ 2909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:33.217023 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:33.217134 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:33.217303 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:33.219593 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.221129 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.223465 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.223914 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.224066 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:33.228888 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:33.228896 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:33.228901 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:33.228950 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:33.232342 (+ 3392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:33.233235 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:33.233243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:33.233247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:33.233418 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:33.234144 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:33.234150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:33.234156 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:33.234477 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:33.234695 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:33.235403 (+ 708us) server_negotiation.cc:299] Negotiation successful
0810 14:38:33.235638 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":282,"threads_started":1}
I20260810 14:38:33.237269 2197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:33.212947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43481 (local address 127.21.55.66:47225)
0810 14:38:33.213909 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:33.213970 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:33.216572 (+ 2602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:33.217641 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:33.217655 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:33.218466 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:33.219339 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:33.219361 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.221343 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.221353 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:33.223299 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:33.223313 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.224174 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.224182 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:33.224520 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:33.225368 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:33.225418 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:33.228609 (+ 3191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:33.232491 (+ 3882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:33.232503 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:33.232526 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:33.233097 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:33.233561 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:33.233570 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:33.233576 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:33.233966 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:33.234659 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:33.234667 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:33.235030 (+ 363us) client_negotiation.cc:769] Sending connection context
0810 14:38:33.235297 (+ 267us) client_negotiation.cc:240] Negotiation successful
0810 14:38:33.235824 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":410,"threads_started":1}
I20260810 14:38:33.259752 2196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:33.244106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:42351 (local address 127.21.55.126:43481)
0810 14:38:33.244374 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:33.244384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:33.244904 (+ 520us) server_negotiation.cc:407] Connection header received
0810 14:38:33.245157 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:33.245166 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:33.245272 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:33.245413 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:33.246393 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.247682 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.250415 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.251094 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.251662 (+ 568us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:33.253371 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:33.253375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:33.253378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:33.253406 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:33.256266 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:33.257329 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:33.257333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:33.257335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:33.257454 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:33.257941 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:33.257945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:33.257947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:33.258432 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:33.258653 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:33.259273 (+ 620us) server_negotiation.cc:299] Negotiation successful
0810 14:38:33.259486 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:38:33.260141 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:33.244119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43481 (local address 127.21.55.66:42351)
0810 14:38:33.244803 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:33.244820 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:33.244931 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:33.245460 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:33.245464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:33.245917 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:33.246239 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:33.246248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.247883 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.247892 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:33.250236 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:33.250257 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.251351 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.251360 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:33.251496 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:33.252392 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:33.252410 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:33.253154 (+ 744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:33.256417 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:33.256426 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:33.256433 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:33.257177 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:33.257595 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:33.257603 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:33.257609 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:33.257816 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:33.258610 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:33.258619 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:33.258906 (+ 287us) client_negotiation.cc:769] Sending connection context
0810 14:38:33.259117 (+ 211us) client_negotiation.cc:240] Negotiation successful
0810 14:38:33.259458 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":372,"threads_started":1}
I20260810 14:38:33.272063 2160 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:41563
I20260810 14:38:33.272082 2402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:41563 every 8 connection(s)
I20260810 14:38:33.273317 2160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:38:33.274868 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 2160
I20260810 14:38:33.275012 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:38:33.275467 21725 external_mini_cluster.cc:1428] Setting key a74b50c70f59e14544a68171eb133864
I20260810 14:38:33.275537 2403 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43481
I20260810 14:38:33.276492 2403 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:33.277297 2403 heartbeater.cc:511] Master 127.21.55.126:43481 requested a full tablet report, sending...
I20260810 14:38:33.279297 1796 ts_manager.cc:194] Registered new tserver with Master: 894207e400bc4b91ad71973a424b3c49 (127.21.55.66:41563)
I20260810 14:38:33.280623 1796 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:47225
WARNING: no policy specified for oryx/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab.
Entry for principal oryx/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab.
I20260810 14:38:33.332526 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43481
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab
--principal=oryx/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:34963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:33.692209 2408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:33.692523 2408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:33.692627 2408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:33.700579 2408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:33.700693 2408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:33.700774 2408 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:38:33.700855 2408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:38:33.710680 2408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab
--principal=oryx/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.55.126:43481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:33.712011 2408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:33.713346 2408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:38:33.721333 2152 heartbeater.cc:503] Master 127.21.55.126:43481 was elected leader, sending a full tablet report...
W20260810 14:38:33.728037 2422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:33.727865 2417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:33.728463 2420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:33.730093 2408 server_base.cc:1034] running on GCE node
I20260810 14:38:33.730906 2408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:33.731963 2408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:33.733264 2408 hybrid_clock.cc:648] HybridClock initialized: now 1786372713733192 us; error 50 us; skew 500 ppm
I20260810 14:38:33.733610 2408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:33 dist-test-slave-h6n0 krb5kdc[1721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372713, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:33.737214 2408 init.cc:377] Logged in from keytab as oryx/127.21.55.67@KRBTEST.COM (short username oryx)
I20260810 14:38:33.738917 2408 webserver.cc:460] Webserver started at http://127.21.55.67:44339/ using document root <none> and password file <none>
I20260810 14:38:33.739753 2408 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:33.739890 2408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:33.740233 2408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:33.743964 2408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7"
format_stamp: "Formatted at 2026-08-10 14:38:33 on dist-test-slave-h6n0"
server_key: "74044836f6926d9eaf67bf96f322622b"
server_key_iv: "e332c102bc1533bf36e9a7a26315c034"
server_key_version: "encryptionkey@0"
I20260810 14:38:33.745184 2408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7"
format_stamp: "Formatted at 2026-08-10 14:38:33 on dist-test-slave-h6n0"
server_key: "74044836f6926d9eaf67bf96f322622b"
server_key_iv: "e332c102bc1533bf36e9a7a26315c034"
server_key_version: "encryptionkey@0"
I20260810 14:38:33.752611 2408 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260810 14:38:33.756635 2433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:33.758011 2408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:38:33.758204 2408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7"
format_stamp: "Formatted at 2026-08-10 14:38:33 on dist-test-slave-h6n0"
server_key: "74044836f6926d9eaf67bf96f322622b"
server_key_iv: "e332c102bc1533bf36e9a7a26315c034"
server_key_version: "encryptionkey@0"
I20260810 14:38:33.758488 2408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:33.779013 2408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:33.783488 2408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:33.783772 2408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:33.786692 2408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:33.786808 2408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:33.786965 2408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:33.787038 2408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:38:33 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372713, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:33.813943 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:33.787533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:59697 (local address 127.21.55.126:43481)
0810 14:38:33.788075 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:33.788082 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:33.789519 (+ 1437us) server_negotiation.cc:407] Connection header received
0810 14:38:33.791755 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:33.791761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:33.791854 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:33.791982 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:33.794448 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.795805 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.799616 (+ 3811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.800091 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.800199 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:33.805979 (+ 5780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:33.805988 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:33.805995 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:33.806060 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:33.810165 (+ 4105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:33.811214 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:33.811221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:33.811226 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:33.811411 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:33.812032 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:33.812036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:33.812038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:33.812313 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:33.812424 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:33.813509 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0810 14:38:33.813672 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":255,"threads_started":1}
I20260810 14:38:33.816324 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:33.788242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43481 (local address 127.21.55.67:59697)
0810 14:38:33.789337 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:33.789388 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:33.791479 (+ 2091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:33.792273 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:33.792281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:33.792940 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:33.794097 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:33.794137 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.796106 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.796121 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:33.799400 (+ 3279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:33.799426 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.800361 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.800371 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:33.800722 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:33.801673 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:33.801711 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:33.805627 (+ 3916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:33.810318 (+ 4691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:33.810337 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:33.810349 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:33.811076 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:33.811546 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:33.811555 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:33.811561 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:33.811881 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:33.812448 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:33.812460 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:33.813019 (+ 559us) client_negotiation.cc:769] Sending connection context
0810 14:38:33.813402 (+ 383us) client_negotiation.cc:240] Negotiation successful
0810 14:38:33.814107 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":360,"threads_started":1}
I20260810 14:38:33.838716 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:33.823811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:48597 (local address 127.21.55.126:43481)
0810 14:38:33.824069 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:33.824077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:33.824675 (+ 598us) server_negotiation.cc:407] Connection header received
0810 14:38:33.824861 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:33.824867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:33.824948 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:33.825056 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:33.825991 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.826812 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.828156 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.828773 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.828915 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:33.832402 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:33.832418 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:33.832426 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:33.832469 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:33.835823 (+ 3354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:33.836705 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:33.836709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:33.836711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:33.836846 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:33.837250 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:33.837257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:33.837263 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:33.837533 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:33.837710 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:33.838096 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 14:38:33.838325 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:38:33.838824 2572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:33.823715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43481 (local address 127.21.55.67:48597)
0810 14:38:33.824559 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:33.824583 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:33.824738 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:33.825080 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:33.825085 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:33.825562 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:33.825848 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:33.825856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.826955 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.826961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:33.827974 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:33.827994 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.829041 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.829056 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:33.829322 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:33.830061 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:33.830093 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:33.831835 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:33.835967 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:33.835973 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:33.835976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:33.836575 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:33.836973 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:33.836978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:33.836982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:33.837122 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:33.837658 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:33.837663 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:33.837853 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 14:38:33.838001 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 14:38:33.838343 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":558,"threads_started":1}
I20260810 14:38:33.850210 2408 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:41387
I20260810 14:38:33.850231 2636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:41387 every 8 connection(s)
I20260810 14:38:33.851572 2408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:38:33.854034 2637 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43481
I20260810 14:38:33.854278 2637 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:33.855003 2637 heartbeater.cc:511] Master 127.21.55.126:43481 requested a full tablet report, sending...
I20260810 14:38:33.856343 1796 ts_manager.cc:194] Registered new tserver with Master: 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387)
I20260810 14:38:33.857167 1796 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:59697
I20260810 14:38:33.860500 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 2408
I20260810 14:38:33.860682 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:38:33.861173 21725 external_mini_cluster.cc:1428] Setting key 5e2e621cdcb847b4854d95bcd9084801
I20260810 14:38:33.865167 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:38:33 dist-test-slave-h6n0 krb5kdc[1721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:38:33 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:33 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:34.284058 2403 heartbeater.cc:503] Master 127.21.55.126:43481 was elected leader, sending a full tablet report...
I20260810 14:38:34.859609 2637 heartbeater.cc:503] Master 127.21.55.126:43481 was elected leader, sending a full tablet report...
W20260810 14:38:35.080449 2655 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.157s user 0.004s sys 0.017s
W20260810 14:38:35.081722 2452 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:33.916817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47056 (local address 127.21.55.126:43481)
0810 14:38:33.917003 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:33.917008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:33.917303 (+ 295us) server_negotiation.cc:407] Connection header received
0810 14:38:33.917506 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:33.917511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:33.917594 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:33.917694 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:33.919156 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.920460 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.922035 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:33.922425 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:33.922524 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.081415 (+1158891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47056: BlockingRecv error: recv got EOF from 127.0.0.1:47056 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 14:38:35 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:35.101794 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.086373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47070 (local address 127.21.55.126:43481)
0810 14:38:35.086604 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:35.086608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:35.086982 (+ 374us) server_negotiation.cc:407] Connection header received
0810 14:38:35.087117 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:35.087121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:35.087179 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:35.087270 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:35.088599 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.089455 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.090855 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.091286 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.091386 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.096011 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:35.096017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:35.096020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:35.096053 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:35.099416 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:35.100081 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:35.100085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:35.100087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:35.100201 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:35.100538 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:35.100541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:35.100543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:35.100808 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:35.101040 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:35.101275 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 14:38:35.101449 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 14:38:35.106182 1796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:38:35.113169 1796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:38:35.161561 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.127156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42172 (local address 127.21.55.66:41563)
0810 14:38:35.152363 (+ 25207us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:35.152375 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:35.152410 (+ 35us) server_negotiation.cc:407] Connection header received
0810 14:38:35.152551 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:35.152559 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:35.152815 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:35.153028 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:35.154357 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.155930 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.159200 (+ 3270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.160380 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.160539 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.160766 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:35.161066 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 14:38:35.161237 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25059,"thread_start_us":24935,"threads_started":1}
I20260810 14:38:35.161633 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.126763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:41563 (local address 127.0.0.1:42172)
0810 14:38:35.150940 (+ 24177us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:35.150997 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:35.151214 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:35.153539 (+ 2325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:35.153549 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:35.153586 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:35.154103 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.154124 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.156078 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.156086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.159018 (+ 2932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.159033 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.160586 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.160592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.160769 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.160826 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 14:38:35.160980 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 14:38:35.161163 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23881,"mutex_wait_us":407,"thread_start_us":27291,"threads_started":1}
I20260810 14:38:35.162313 2679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.127725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52044 (local address 127.21.55.67:41387)
0810 14:38:35.152721 (+ 24996us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:35.152729 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:35.152755 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:38:35.152855 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:35.152865 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:35.153061 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:35.153262 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:35.154951 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.156468 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.159239 (+ 2771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.160512 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.160645 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.160847 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:35.161827 (+ 980us) server_negotiation.cc:299] Negotiation successful
0810 14:38:35.162025 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24840,"thread_start_us":26104,"threads_started":1}
I20260810 14:38:35.162616 2673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.126968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:41387 (local address 127.0.0.1:52044)
0810 14:38:35.151715 (+ 24747us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:35.151736 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:35.151860 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:35.154119 (+ 2259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:35.154125 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:35.154160 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:35.154769 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.154782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.156629 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.156635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.159018 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.159033 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.160936 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.160945 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.161113 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.161143 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 14:38:35.161893 (+ 750us) client_negotiation.cc:240] Negotiation successful
0810 14:38:35.162037 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24568,"mutex_wait_us":50,"thread_start_us":26217,"threads_started":1}
I20260810 14:38:35.163470 2672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.127331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:40573 (local address 127.0.0.1:54704)
0810 14:38:35.154439 (+ 27108us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:35.154462 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:35.154608 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:35.155522 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:35.155527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:35.155550 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:35.155936 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.155948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.157385 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.157396 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.159635 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.159654 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.162653 (+ 2999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.162665 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.162858 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.162901 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 14:38:35.162996 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 14:38:35.163121 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26948,"mutex_wait_us":80,"thread_start_us":23132,"threads_started":1}
I20260810 14:38:35.163538 2677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.127031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.21.55.65:40573)
0810 14:38:35.153883 (+ 26852us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:35.153894 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:35.154762 (+ 868us) server_negotiation.cc:407] Connection header received
0810 14:38:35.154873 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:35.154880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:35.155114 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:35.155302 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:35.156130 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.157228 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.159819 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.161276 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.161412 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.161612 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:35.163066 (+ 1454us) server_negotiation.cc:299] Negotiation successful
0810 14:38:35.163238 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26685,"thread_start_us":26510,"threads_started":1}
I20260810 14:38:35.166261 2272 tablet_service.cc:1467] Processing CreateTablet for tablet 0541be0ab32f4b389afcf7cf55437a17 (DEFAULT_TABLE table=test-table [id=1e2ae303dc2e4f24bfc94d455d1e0017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:35.168048 2272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0541be0ab32f4b389afcf7cf55437a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:35.168076 2034 tablet_service.cc:1467] Processing CreateTablet for tablet 0541be0ab32f4b389afcf7cf55437a17 (DEFAULT_TABLE table=test-table [id=1e2ae303dc2e4f24bfc94d455d1e0017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:35.168011 2526 tablet_service.cc:1467] Processing CreateTablet for tablet 0541be0ab32f4b389afcf7cf55437a17 (DEFAULT_TABLE table=test-table [id=1e2ae303dc2e4f24bfc94d455d1e0017]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:35.169512 2526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0541be0ab32f4b389afcf7cf55437a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:35.169539 2034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0541be0ab32f4b389afcf7cf55437a17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:35.181699 2681 tablet_bootstrap.cc:492] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: Bootstrap starting.
I20260810 14:38:35.184214 2683 tablet_bootstrap.cc:492] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Bootstrap starting.
I20260810 14:38:35.187865 2681 tablet_bootstrap.cc:654] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:35.188325 2683 tablet_bootstrap.cc:654] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:35.189311 2681 log.cc:824] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:35.189496 2683 log.cc:824] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:35.190456 2686 tablet_bootstrap.cc:492] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: Bootstrap starting.
I20260810 14:38:35.192406 2681 tablet_bootstrap.cc:492] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: No bootstrap required, opened a new log
I20260810 14:38:35.192639 2683 tablet_bootstrap.cc:492] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: No bootstrap required, opened a new log
I20260810 14:38:35.192719 2681 ts_tablet_manager.cc:1397] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260810 14:38:35.192910 2683 ts_tablet_manager.cc:1397] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260810 14:38:35.194941 2686 tablet_bootstrap.cc:654] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:35.196184 2686 log.cc:824] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:35.200335 2686 tablet_bootstrap.cc:492] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: No bootstrap required, opened a new log
I20260810 14:38:35.200631 2686 ts_tablet_manager.cc:1397] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 14:38:35.201038 2681 raft_consensus.cc:348] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.201416 2681 raft_consensus.cc:374] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:35.201534 2681 raft_consensus.cc:729] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894207e400bc4b91ad71973a424b3c49, State: Initialized, Role: FOLLOWER
I20260810 14:38:35.202157 2681 consensus_queue.cc:260] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.204003 2681 ts_tablet_manager.cc:1428] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260810 14:38:35.204535 2683 raft_consensus.cc:348] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.205020 2683 raft_consensus.cc:374] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:35.205147 2683 raft_consensus.cc:729] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac0b4c39b9449e09c94c9c4c8b838a7, State: Initialized, Role: FOLLOWER
I20260810 14:38:35.205768 2683 consensus_queue.cc:260] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.207646 2683 ts_tablet_manager.cc:1428] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260810 14:38:35.211580 2686 raft_consensus.cc:348] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.211968 2686 raft_consensus.cc:374] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:35.212097 2686 raft_consensus.cc:729] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04da3915c6bf4d5f9cd132f484029a80, State: Initialized, Role: FOLLOWER
I20260810 14:38:35.212636 2686 consensus_queue.cc:260] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.214380 2686 ts_tablet_manager.cc:1428] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
W20260810 14:38:35.276808 2404 tablet.cc:2369] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:38:35.354475 2639 tablet.cc:2369] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:38:35.429332 2691 raft_consensus.cc:482] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:38:35.429698 2691 raft_consensus.cc:504] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.431488 2691 leader_election.cc:288] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573), 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387)
I20260810 14:38:35.438216 2677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.431926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:52301 (local address 127.21.55.65:40573)
0810 14:38:35.432165 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:35.432172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:35.432930 (+ 758us) server_negotiation.cc:407] Connection header received
0810 14:38:35.433182 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:35.433187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:35.433245 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:35.433343 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:35.433879 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.434677 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.436492 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.437266 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.437460 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.437542 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:35.437830 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 14:38:35.437955 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:38:35.438328 2695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.431852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:40573 (local address 127.21.55.66:52301)
0810 14:38:35.432813 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:35.432831 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:35.433021 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:35.433365 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:35.433369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:35.433403 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:35.433734 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.433745 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.434839 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.434842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.436332 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.436345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.437490 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.437496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.437662 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.437697 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 14:38:35.437803 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 14:38:35.437931 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":635,"threads_started":1}
I20260810 14:38:35.438359 2679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.432018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:52561 (local address 127.21.55.67:41387)
0810 14:38:35.432250 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:35.432259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:35.433283 (+ 1024us) server_negotiation.cc:407] Connection header received
0810 14:38:35.433496 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:35.433501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:35.433558 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:35.433657 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:35.434188 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.434962 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.436690 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.437446 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.437627 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.437705 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:35.438000 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 14:38:35.438122 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 14:38:35.438508 2696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.431904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:41387 (local address 127.21.55.66:52561)
0810 14:38:35.433176 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:35.433191 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:35.433353 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:35.433676 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:35.433680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:35.433707 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:35.434041 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.434049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.435100 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.435106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.436521 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.436534 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.437670 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.437676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.437828 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.437851 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 14:38:35.437959 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 14:38:35.438084 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1166,"mutex_wait_us":3,"thread_start_us":1021,"threads_started":1}
I20260810 14:38:35.439491 2077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0541be0ab32f4b389afcf7cf55437a17" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04da3915c6bf4d5f9cd132f484029a80" is_pre_election: true
I20260810 14:38:35.439640 2560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0541be0ab32f4b389afcf7cf55437a17" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" is_pre_election: true
I20260810 14:38:35.440042 2077 raft_consensus.cc:2454] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 0.
I20260810 14:38:35.440176 2560 raft_consensus.cc:2454] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 0.
I20260810 14:38:35.440697 2188 leader_election.cc:302] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04da3915c6bf4d5f9cd132f484029a80, 894207e400bc4b91ad71973a424b3c49; no voters:
I20260810 14:38:35.441082 2691 raft_consensus.cc:2781] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:38:35.441242 2691 raft_consensus.cc:482] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:38:35.441358 2691 raft_consensus.cc:3037] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:35.443583 2691 raft_consensus.cc:504] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.444240 2691 leader_election.cc:288] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 election: Requested vote from peers 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573), 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387)
I20260810 14:38:35.444720 2077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0541be0ab32f4b389afcf7cf55437a17" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04da3915c6bf4d5f9cd132f484029a80"
I20260810 14:38:35.444808 2560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0541be0ab32f4b389afcf7cf55437a17" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7"
I20260810 14:38:35.444927 2077 raft_consensus.cc:3037] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:35.445067 2560 raft_consensus.cc:3037] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:35.447278 2077 raft_consensus.cc:2454] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 1.
I20260810 14:38:35.447389 2560 raft_consensus.cc:2454] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 1.
I20260810 14:38:35.448055 2188 leader_election.cc:302] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04da3915c6bf4d5f9cd132f484029a80, 894207e400bc4b91ad71973a424b3c49; no voters:
I20260810 14:38:35.448632 2691 raft_consensus.cc:2781] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:35.449343 2691 raft_consensus.cc:686] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 1 LEADER]: Becoming Leader. State: Replica: 894207e400bc4b91ad71973a424b3c49, State: Running, Role: LEADER
I20260810 14:38:35.450146 2691 consensus_queue.cc:237] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.455495 1795 catalog_manager.cc:5495] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 894207e400bc4b91ad71973a424b3c49 (127.21.55.66). New cstate: current_term: 1 leader_uuid: "894207e400bc4b91ad71973a424b3c49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } health_report { overall_health: UNKNOWN } } }
W20260810 14:38:35.464716 2153 tablet.cc:2369] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:38:35.536631 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.530823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42176 (local address 127.21.55.66:41563)
0810 14:38:35.531066 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:35.531076 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:35.531174 (+ 98us) server_negotiation.cc:407] Connection header received
0810 14:38:35.531450 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:35.531455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:35.531542 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:35.531721 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:38:35.532114 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.532839 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.534291 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.534778 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.534936 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.535424 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:38:35.535977 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:38:35.536143 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:35.536238 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 14:38:35.536329 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:38:35.545182 2560 raft_consensus.cc:1261] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Refusing update from remote peer 894207e400bc4b91ad71973a424b3c49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:35.545188 2077 raft_consensus.cc:1261] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Refusing update from remote peer 894207e400bc4b91ad71973a424b3c49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:35.546164 2698 consensus_queue.cc:1035] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:35.546607 2691 consensus_queue.cc:1035] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:35.572974 1796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:35.573269 1796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:35.576849 1796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6: Sending DeleteTablet for 3 replicas of tablet 0541be0ab32f4b389afcf7cf55437a17
I20260810 14:38:35.578084 2272 tablet_service.cc:1514] Processing DeleteTablet for tablet 0541be0ab32f4b389afcf7cf55437a17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:35 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:42172
I20260810 14:38:35.578089 2034 tablet_service.cc:1514] Processing DeleteTablet for tablet 0541be0ab32f4b389afcf7cf55437a17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:35 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:54704
I20260810 14:38:35.578176 2526 tablet_service.cc:1514] Processing DeleteTablet for tablet 0541be0ab32f4b389afcf7cf55437a17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:35 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:52044
I20260810 14:38:35.583673 1796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:38:35.584909 1796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:38:35.588742 2721 tablet_replica.cc:330] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: stopping tablet replica
I20260810 14:38:35.589315 2721 raft_consensus.cc:2229] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:38:35.589985 2721 raft_consensus.cc:2258] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:35.591095 2722 tablet_replica.cc:330] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: stopping tablet replica
I20260810 14:38:35.591559 2722 raft_consensus.cc:2229] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:35.592077 2722 raft_consensus.cc:2258] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:35.592567 2721 ts_tablet_manager.cc:1905] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:35.596194 2720 tablet_replica.cc:330] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: stopping tablet replica
I20260810 14:38:35.596671 2720 raft_consensus.cc:2229] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:35.596809 2526 tablet_service.cc:1467] Processing CreateTablet for tablet 9829f1c14b36473593001ece1cb1e390 (DEFAULT_TABLE table=test-table [id=47e26288648c4ca893599501f33347f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:35.596771 2034 tablet_service.cc:1467] Processing CreateTablet for tablet 9829f1c14b36473593001ece1cb1e390 (DEFAULT_TABLE table=test-table [id=47e26288648c4ca893599501f33347f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:35.597263 2526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9829f1c14b36473593001ece1cb1e390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:35.597302 2034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9829f1c14b36473593001ece1cb1e390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:35.597374 2720 raft_consensus.cc:2258] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:35.602392 2721 ts_tablet_manager.cc:1918] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:35.602787 2721 log.cc:1196] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/0541be0ab32f4b389afcf7cf55437a17
I20260810 14:38:35.603958 2686 tablet_bootstrap.cc:492] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80: Bootstrap starting.
I20260810 14:38:35.604914 2683 tablet_bootstrap.cc:492] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Bootstrap starting.
I20260810 14:38:35.605684 2686 tablet_bootstrap.cc:654] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:35.606565 2683 tablet_bootstrap.cc:654] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:35.607378 2720 ts_tablet_manager.cc:1905] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:35.609360 2722 ts_tablet_manager.cc:1905] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:35.611729 2683 tablet_bootstrap.cc:492] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7: No bootstrap required, opened a new log
I20260810 14:38:35.611922 2683 ts_tablet_manager.cc:1397] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260810 14:38:35.613016 2683 raft_consensus.cc:348] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.613389 2683 raft_consensus.cc:374] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:35.613535 2683 raft_consensus.cc:729] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac0b4c39b9449e09c94c9c4c8b838a7, State: Initialized, Role: FOLLOWER
I20260810 14:38:35.613829 2683 consensus_queue.cc:260] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.614400 2683 ts_tablet_manager.cc:1428] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260810 14:38:35.617493 2720 ts_tablet_manager.cc:1918] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:35.617938 2722 ts_tablet_manager.cc:1918] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:35.618120 2722 log.cc:1196] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/0541be0ab32f4b389afcf7cf55437a17
I20260810 14:38:35.605489 2272 tablet_service.cc:1467] Processing CreateTablet for tablet 9829f1c14b36473593001ece1cb1e390 (DEFAULT_TABLE table=test-table [id=47e26288648c4ca893599501f33347f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:35.618975 2721 ts_tablet_manager.cc:1939] T 0541be0ab32f4b389afcf7cf55437a17 P 894207e400bc4b91ad71973a424b3c49: Deleting consensus metadata
I20260810 14:38:35.621083 1768 catalog_manager.cc:4841] TS 894207e400bc4b91ad71973a424b3c49 (127.21.55.66:41563): tablet 0541be0ab32f4b389afcf7cf55437a17 (table test-table [id=1e2ae303dc2e4f24bfc94d455d1e0017]) successfully deleted
I20260810 14:38:35.621207 2272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9829f1c14b36473593001ece1cb1e390. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:35.623001 2722 ts_tablet_manager.cc:1939] T 0541be0ab32f4b389afcf7cf55437a17 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting consensus metadata
I20260810 14:38:35.624694 1771 catalog_manager.cc:4841] TS 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387): tablet 0541be0ab32f4b389afcf7cf55437a17 (table test-table [id=1e2ae303dc2e4f24bfc94d455d1e0017]) successfully deleted
I20260810 14:38:35.624675 2686 tablet_bootstrap.cc:492] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80: No bootstrap required, opened a new log
I20260810 14:38:35.624889 2686 ts_tablet_manager.cc:1397] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.005s
I20260810 14:38:35.625726 2686 raft_consensus.cc:348] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.626003 2686 raft_consensus.cc:374] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:35.626119 2686 raft_consensus.cc:729] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04da3915c6bf4d5f9cd132f484029a80, State: Initialized, Role: FOLLOWER
I20260810 14:38:35.626766 2686 consensus_queue.cc:260] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.628538 2681 tablet_bootstrap.cc:492] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49: Bootstrap starting.
I20260810 14:38:35.628567 2686 ts_tablet_manager.cc:1428] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 14:38:35.628929 2720 log.cc:1196] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/0541be0ab32f4b389afcf7cf55437a17
I20260810 14:38:35.629900 2720 ts_tablet_manager.cc:1939] T 0541be0ab32f4b389afcf7cf55437a17 P 04da3915c6bf4d5f9cd132f484029a80: Deleting consensus metadata
I20260810 14:38:35.630435 2681 tablet_bootstrap.cc:654] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:35.633205 1770 catalog_manager.cc:4841] TS 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573): tablet 0541be0ab32f4b389afcf7cf55437a17 (table test-table [id=1e2ae303dc2e4f24bfc94d455d1e0017]) successfully deleted
I20260810 14:38:35.634745 2681 tablet_bootstrap.cc:492] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49: No bootstrap required, opened a new log
I20260810 14:38:35.634977 2681 ts_tablet_manager.cc:1397] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 14:38:35.635787 2681 raft_consensus.cc:348] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.635975 2681 raft_consensus.cc:374] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:35.636106 2681 raft_consensus.cc:729] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894207e400bc4b91ad71973a424b3c49, State: Initialized, Role: FOLLOWER
I20260810 14:38:35.636394 2681 consensus_queue.cc:260] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.636627 2698 raft_consensus.cc:482] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:38:35.636829 2698 raft_consensus.cc:504] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.637053 2681 ts_tablet_manager.cc:1428] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:38:35.637538 2698 leader_election.cc:288] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573), 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387)
I20260810 14:38:35.637980 2077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9829f1c14b36473593001ece1cb1e390" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04da3915c6bf4d5f9cd132f484029a80" is_pre_election: true
I20260810 14:38:35.638242 2077 raft_consensus.cc:2454] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 0.
I20260810 14:38:35.638219 2560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9829f1c14b36473593001ece1cb1e390" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" is_pre_election: true
I20260810 14:38:35.638486 2560 raft_consensus.cc:2454] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 0.
I20260810 14:38:35.638737 2188 leader_election.cc:302] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04da3915c6bf4d5f9cd132f484029a80, 894207e400bc4b91ad71973a424b3c49; no voters:
I20260810 14:38:35.639158 2698 raft_consensus.cc:2781] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:38:35.639272 2698 raft_consensus.cc:482] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:38:35.639385 2698 raft_consensus.cc:3037] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:35.642603 2698 raft_consensus.cc:504] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.643862 2698 leader_election.cc:288] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 election: Requested vote from peers 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573), 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387)
I20260810 14:38:35.644658 2560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9829f1c14b36473593001ece1cb1e390" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7"
I20260810 14:38:35.644924 2560 raft_consensus.cc:3037] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:35.644948 2077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9829f1c14b36473593001ece1cb1e390" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04da3915c6bf4d5f9cd132f484029a80"
I20260810 14:38:35.645170 2077 raft_consensus.cc:3037] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:35.649009 2560 raft_consensus.cc:2454] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 1.
I20260810 14:38:35.649008 2077 raft_consensus.cc:2454] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 1.
I20260810 14:38:35.649839 2190 leader_election.cc:302] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ac0b4c39b9449e09c94c9c4c8b838a7, 894207e400bc4b91ad71973a424b3c49; no voters:
I20260810 14:38:35.650169 2698 raft_consensus.cc:2781] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:35.650454 2698 raft_consensus.cc:686] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 1 LEADER]: Becoming Leader. State: Replica: 894207e400bc4b91ad71973a424b3c49, State: Running, Role: LEADER
I20260810 14:38:35.650753 2698 consensus_queue.cc:237] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.653352 1796 catalog_manager.cc:5495] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 894207e400bc4b91ad71973a424b3c49 (127.21.55.66). New cstate: current_term: 1 leader_uuid: "894207e400bc4b91ad71973a424b3c49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } health_report { overall_health: UNKNOWN } } }
Aug 10 14:38:35 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372711, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.126@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:35.701453 2737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.686566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.21.55.126:43481)
0810 14:38:35.687338 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:35.687345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:35.687568 (+ 223us) server_negotiation.cc:407] Connection header received
0810 14:38:35.687775 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:35.687782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:35.687841 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:35.687957 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:35.689046 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.689800 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.691146 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.691545 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.691674 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.695122 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:35.695127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:35.695130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:35.695156 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:35.697860 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:35.698675 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:35.698681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:35.698684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:35.698810 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:35.699190 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:35.699195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:35.699198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:35.699467 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:35.699617 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:35.700458 (+ 841us) server_negotiation.cc:299] Negotiation successful
0810 14:38:35.700911 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":219,"server-negotiator.queue_time_us":585,"thread_start_us":399,"threads_started":1}
I20260810 14:38:35.701454 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.686585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43481 (local address 127.0.0.1:47078)
0810 14:38:35.687467 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:35.687483 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:35.687647 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:35.687982 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:35.687986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:35.688716 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:35.688909 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.688919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.689955 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.689960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.691005 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.691017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.691717 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.691723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.691905 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.692629 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:35.692652 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:35.694921 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:35.698051 (+ 3130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:35.698065 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:35.698073 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:35.698520 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:35.698938 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:35.698943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:35.698947 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:35.699076 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:35.699608 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:35.699616 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:35.699779 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 14:38:35.700314 (+ 535us) client_negotiation.cc:240] Negotiation successful
0810 14:38:35.700768 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":617,"threads_started":1}
I20260810 14:38:35.709030 1796 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:47078:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 14:38:35.710364 1796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:38:35.719861 2272 tablet_service.cc:1467] Processing CreateTablet for tablet 3f13c183eca5473d84ed980df95d06f8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c68adca6933b4448ab43a3ecb75ba491]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:38:35.719671 2034 tablet_service.cc:1467] Processing CreateTablet for tablet 3f13c183eca5473d84ed980df95d06f8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c68adca6933b4448ab43a3ecb75ba491]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:38:35.720557 2272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f13c183eca5473d84ed980df95d06f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:35.720703 2034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f13c183eca5473d84ed980df95d06f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:35.723102 2526 tablet_service.cc:1467] Processing CreateTablet for tablet 3f13c183eca5473d84ed980df95d06f8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c68adca6933b4448ab43a3ecb75ba491]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:38:35.723668 2526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f13c183eca5473d84ed980df95d06f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:35.730209 2686 tablet_bootstrap.cc:492] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: Bootstrap starting.
I20260810 14:38:35.732267 2686 tablet_bootstrap.cc:654] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:35.738056 2683 tablet_bootstrap.cc:492] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Bootstrap starting.
I20260810 14:38:35.739866 2683 tablet_bootstrap.cc:654] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:35.741598 2681 tablet_bootstrap.cc:492] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49: Bootstrap starting.
I20260810 14:38:35.742630 2683 tablet_bootstrap.cc:492] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7: No bootstrap required, opened a new log
I20260810 14:38:35.742861 2683 ts_tablet_manager.cc:1397] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 14:38:35.743656 2681 tablet_bootstrap.cc:654] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:35.743666 2683 raft_consensus.cc:348] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.743920 2683 raft_consensus.cc:374] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:35.743992 2683 raft_consensus.cc:729] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac0b4c39b9449e09c94c9c4c8b838a7, State: Initialized, Role: FOLLOWER
I20260810 14:38:35.744284 2686 tablet_bootstrap.cc:492] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: No bootstrap required, opened a new log
I20260810 14:38:35.744290 2683 consensus_queue.cc:260] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.744459 2686 ts_tablet_manager.cc:1397] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260810 14:38:35.744745 2692 tablet_replica.cc:439] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } } }
I20260810 14:38:35.745080 2692 tablet_replica.cc:442] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:38:35.745345 2686 raft_consensus.cc:348] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.746253 2686 raft_consensus.cc:374] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:35.746377 2686 raft_consensus.cc:729] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04da3915c6bf4d5f9cd132f484029a80, State: Initialized, Role: FOLLOWER
I20260810 14:38:35.747159 2686 consensus_queue.cc:260] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.748046 2686 ts_tablet_manager.cc:1428] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:38:35.747884 2694 tablet_replica.cc:439] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } } }
I20260810 14:38:35.748315 2694 tablet_replica.cc:442] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:38:35.749732 2683 ts_tablet_manager.cc:1428] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260810 14:38:35.753201 2681 tablet_bootstrap.cc:492] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49: No bootstrap required, opened a new log
I20260810 14:38:35.753391 2681 ts_tablet_manager.cc:1397] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260810 14:38:35.753991 2681 raft_consensus.cc:348] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.754204 2681 raft_consensus.cc:374] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:35.754310 2681 raft_consensus.cc:729] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894207e400bc4b91ad71973a424b3c49, State: Initialized, Role: FOLLOWER
I20260810 14:38:35.754616 2681 consensus_queue.cc:260] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.755108 2698 tablet_replica.cc:439] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } } }
I20260810 14:38:35.755267 2681 ts_tablet_manager.cc:1428] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:38:35.755416 2698 tablet_replica.cc:442] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:38:35.969399 2694 raft_consensus.cc:482] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:38:35.969741 2694 raft_consensus.cc:504] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.972339 2694 leader_election.cc:288] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 894207e400bc4b91ad71973a424b3c49 (127.21.55.66:41563), 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387)
I20260810 14:38:35.983512 2745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.972572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:41563 (local address 127.21.55.65:36611)
0810 14:38:35.973528 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:35.973551 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:35.973787 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:35.974375 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:35.974380 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:35.974408 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:35.974836 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.974851 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.976942 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.976951 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.980231 (+ 3280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.980250 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.982479 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.982485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.982726 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.982763 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 14:38:35.982863 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 14:38:35.983033 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"thread_start_us":697,"threads_started":1}
I20260810 14:38:35.983763 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.972897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:36611 (local address 127.21.55.66:41563)
0810 14:38:35.973132 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:35.973140 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:35.973648 (+ 508us) server_negotiation.cc:407] Connection header received
0810 14:38:35.973975 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:35.973981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:35.974085 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:35.974156 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:35.975058 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.976387 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.980978 (+ 4591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.982302 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.982495 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.982610 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:35.983294 (+ 684us) server_negotiation.cc:299] Negotiation successful
0810 14:38:35.983445 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:38:35.984758 2315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f13c183eca5473d84ed980df95d06f8" candidate_uuid: "04da3915c6bf4d5f9cd132f484029a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "894207e400bc4b91ad71973a424b3c49" is_pre_election: true
I20260810 14:38:35.985250 2315 raft_consensus.cc:2454] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04da3915c6bf4d5f9cd132f484029a80 in term 0.
I20260810 14:38:35.985924 1949 leader_election.cc:302] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04da3915c6bf4d5f9cd132f484029a80, 894207e400bc4b91ad71973a424b3c49; no voters:
I20260810 14:38:35.986402 2694 raft_consensus.cc:2781] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:38:35.986563 2694 raft_consensus.cc:482] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:38:35.986680 2694 raft_consensus.cc:3037] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:35.988507 2747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.972935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:49351 (local address 127.21.55.67:41387)
0810 14:38:35.980841 (+ 7906us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:35.980847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:35.981442 (+ 595us) server_negotiation.cc:407] Connection header received
0810 14:38:35.981666 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:35.981669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:35.981729 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:35.981842 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:38:35.982604 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.983887 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.986326 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.987320 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.987548 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.987630 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:35.988038 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 14:38:35.988194 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7781,"thread_start_us":7703,"threads_started":1}
I20260810 14:38:35.988543 2752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:35.972841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:41387 (local address 127.21.55.65:49351)
0810 14:38:35.981327 (+ 8486us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:35.981349 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:35.981508 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:35.981862 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:35.981866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:35.981891 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:38:35.982312 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.982333 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.984068 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.984074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.986130 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:35.986143 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:35.987603 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:35.987609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:35.987814 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:35.987866 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 14:38:35.988012 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 14:38:35.988176 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8310,"thread_start_us":8162,"threads_started":1}
I20260810 14:38:35.989279 2560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f13c183eca5473d84ed980df95d06f8" candidate_uuid: "04da3915c6bf4d5f9cd132f484029a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" is_pre_election: true
I20260810 14:38:35.989554 2560 raft_consensus.cc:2454] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04da3915c6bf4d5f9cd132f484029a80 in term 0.
I20260810 14:38:35.990216 2694 raft_consensus.cc:504] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:35.991065 2694 leader_election.cc:288] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [CANDIDATE]: Term 1 election: Requested vote from peers 894207e400bc4b91ad71973a424b3c49 (127.21.55.66:41563), 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387)
I20260810 14:38:35.991463 2560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f13c183eca5473d84ed980df95d06f8" candidate_uuid: "04da3915c6bf4d5f9cd132f484029a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7"
I20260810 14:38:35.991550 2315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f13c183eca5473d84ed980df95d06f8" candidate_uuid: "04da3915c6bf4d5f9cd132f484029a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "894207e400bc4b91ad71973a424b3c49"
I20260810 14:38:35.991716 2560 raft_consensus.cc:3037] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:35.991780 2315 raft_consensus.cc:3037] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:35.994813 2560 raft_consensus.cc:2454] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04da3915c6bf4d5f9cd132f484029a80 in term 1.
I20260810 14:38:35.995255 2315 raft_consensus.cc:2454] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04da3915c6bf4d5f9cd132f484029a80 in term 1.
I20260810 14:38:35.995538 1955 leader_election.cc:302] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04da3915c6bf4d5f9cd132f484029a80, 0ac0b4c39b9449e09c94c9c4c8b838a7; no voters:
I20260810 14:38:35.996006 2694 raft_consensus.cc:2781] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:35.996518 2694 raft_consensus.cc:686] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 LEADER]: Becoming Leader. State: Replica: 04da3915c6bf4d5f9cd132f484029a80, State: Running, Role: LEADER
I20260810 14:38:35.997264 2694 consensus_queue.cc:237] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.000067 2755 tablet_replica.cc:439] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: TxnStatusTablet state changed. Reason: New leader 04da3915c6bf4d5f9cd132f484029a80. Latest consensus state: current_term: 1 leader_uuid: "04da3915c6bf4d5f9cd132f484029a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } } }
I20260810 14:38:36.001422 2755 tablet_replica.cc:442] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:38:36.001754 1796 catalog_manager.cc:5495] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65). New cstate: current_term: 1 leader_uuid: "04da3915c6bf4d5f9cd132f484029a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } health_report { overall_health: UNKNOWN } } }
I20260810 14:38:36.002388 2757 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 14:38:36.002662 2757 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 14:38:36.049641 2698 consensus_queue.cc:1035] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:36.073158 2698 consensus_queue.cc:1035] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:36.079173 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:36.060728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:40573 (local address 127.0.0.1:54716)
0810 14:38:36.060997 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:36.061022 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:36.061236 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:36.070001 (+ 8765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:36.070007 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:36.070022 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:38:36.070228 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:36.070238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.072528 (+ 2290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.072534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:36.074228 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:36.074244 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.075955 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.075962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:36.076166 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:36.076255 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:38:36.078107 (+ 1852us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:38:36.078155 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 14:38:36.078288 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 14:38:36.078579 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 14:38:36.080854 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:36.057807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54716 (local address 127.21.55.65:40573)
0810 14:38:36.069371 (+ 11564us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:36.069380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:36.069410 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:38:36.069519 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:36.069525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:36.069651 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:36.070371 (+ 720us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:38:36.070738 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.072368 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.075013 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.075782 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.076431 (+ 649us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:36.076562 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:38:36.077832 (+ 1270us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:38:36.080119 (+ 2287us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:36.080203 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 14:38:36.080412 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11395,"thread_start_us":11999,"threads_started":1}
I20260810 14:38:36.088904 2315 raft_consensus.cc:1261] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Refusing update from remote peer 04da3915c6bf4d5f9cd132f484029a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:36.089407 2558 raft_consensus.cc:1261] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Refusing update from remote peer 04da3915c6bf4d5f9cd132f484029a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:36.090338 2755 consensus_queue.cc:1035] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 14:38:36.091225 2694 consensus_queue.cc:1035] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:36.102829 2715 tablet_replica.cc:439] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49: TxnStatusTablet state changed. Reason: New leader 04da3915c6bf4d5f9cd132f484029a80. Latest consensus state: current_term: 1 leader_uuid: "04da3915c6bf4d5f9cd132f484029a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } } }
I20260810 14:38:36.103258 2715 tablet_replica.cc:442] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:38:36.104277 2692 tablet_replica.cc:439] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7: TxnStatusTablet state changed. Reason: New leader 04da3915c6bf4d5f9cd132f484029a80. Latest consensus state: current_term: 1 leader_uuid: "04da3915c6bf4d5f9cd132f484029a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } } }
I20260810 14:38:36.104568 2692 tablet_replica.cc:442] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:38:36.109736 2769 mvcc.cc:205] Tried to move back new op lower bound from 7316982645088534528 to 7316982644729417728. Current Snapshot: MvccSnapshot[applied={T|T < 7316982645088534528}]
I20260810 14:38:36.114073 2755 tablet_replica.cc:439] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04da3915c6bf4d5f9cd132f484029a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } } }
I20260810 14:38:36.114410 2755 tablet_replica.cc:442] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:38:36.116753 2779 tablet_replica.cc:439] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04da3915c6bf4d5f9cd132f484029a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } } }
I20260810 14:38:36.117296 2779 tablet_replica.cc:442] T 3f13c183eca5473d84ed980df95d06f8 P 04da3915c6bf4d5f9cd132f484029a80: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:38:36.129707 2692 tablet_replica.cc:439] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04da3915c6bf4d5f9cd132f484029a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } } }
I20260810 14:38:36.130002 2692 tablet_replica.cc:442] T 3f13c183eca5473d84ed980df95d06f8 P 0ac0b4c39b9449e09c94c9c4c8b838a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:38:36.135195 2715 tablet_replica.cc:439] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04da3915c6bf4d5f9cd132f484029a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } } }
I20260810 14:38:36.135470 2715 tablet_replica.cc:442] T 3f13c183eca5473d84ed980df95d06f8 P 894207e400bc4b91ad71973a424b3c49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:38:36.151559 2763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:36.145861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:50455 (local address 127.21.55.65:40573)
0810 14:38:36.146136 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:36.146144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:36.146432 (+ 288us) server_negotiation.cc:407] Connection header received
0810 14:38:36.146721 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:36.146726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:36.146797 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:36.146899 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:38:36.147419 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.148267 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.149630 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.150077 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.150239 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:36.150753 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:38:36.150858 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:38:36.150966 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:36.151115 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 14:38:36.151200 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 14:38:36.151628 2789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:36.145645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:40573 (local address 127.21.55.66:50455)
0810 14:38:36.146291 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:36.146311 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:36.146508 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:36.146919 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:36.146923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:36.146952 (+ 29us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:38:36.147252 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:36.147261 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.148424 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.148429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:36.149462 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:36.149474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.150312 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.150324 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:36.150434 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:36.150491 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:38:36.150966 (+ 475us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:38:36.150981 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 14:38:36.151087 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 14:38:36.151221 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":330,"threads_started":1}
I20260810 14:38:36.199366 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:36.192487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:48889 (local address 127.21.55.66:41563)
0810 14:38:36.192657 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:36.192666 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:36.193384 (+ 718us) server_negotiation.cc:407] Connection header received
0810 14:38:36.193707 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:36.193712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:36.193802 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:36.193883 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:38:36.194615 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.195404 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.197317 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.197971 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.198125 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:36.198600 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:38:36.198712 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:38:36.198814 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:36.198973 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 14:38:36.199053 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 14:38:36.199559 2793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:36.192527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:41563 (local address 127.21.55.65:48889)
0810 14:38:36.193279 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:36.193301 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:36.193498 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:36.194073 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:36.194076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:36.194086 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:38:36.194421 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:36.194428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.195544 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.195549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:36.197182 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:36.197193 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.198176 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.198184 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:36.198330 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:36.198388 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:38:36.198818 (+ 430us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:38:36.198833 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 14:38:36.198952 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 14:38:36.199086 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":310,"threads_started":1}
I20260810 14:38:36.237929 1796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:36.238224 1796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:36.242278 1796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6: Sending DeleteTablet for 3 replicas of tablet 9829f1c14b36473593001ece1cb1e390
I20260810 14:38:36.243446 2272 tablet_service.cc:1514] Processing DeleteTablet for tablet 9829f1c14b36473593001ece1cb1e390 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:36 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:42172
I20260810 14:38:36.243916 2526 tablet_service.cc:1514] Processing DeleteTablet for tablet 9829f1c14b36473593001ece1cb1e390 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:36 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:52044
I20260810 14:38:36.244014 2034 tablet_service.cc:1514] Processing DeleteTablet for tablet 9829f1c14b36473593001ece1cb1e390 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:36 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:54704
I20260810 14:38:36.249966 2805 tablet_replica.cc:330] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7: stopping tablet replica
I20260810 14:38:36.250134 2807 tablet_replica.cc:330] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80: stopping tablet replica
I20260810 14:38:36.250384 2805 raft_consensus.cc:2229] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:36.250371 2807 raft_consensus.cc:2229] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:36.250731 2807 raft_consensus.cc:2258] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:36.250775 2805 raft_consensus.cc:2258] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:36.251355 2802 tablet_replica.cc:330] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49: stopping tablet replica
I20260810 14:38:36.251592 2802 raft_consensus.cc:2229] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:38:36.252058 2802 raft_consensus.cc:2258] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:36.252841 2805 ts_tablet_manager.cc:1905] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:36.253072 2807 ts_tablet_manager.cc:1905] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:36.261829 2805 ts_tablet_manager.cc:1918] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:38:36.261951 2807 ts_tablet_manager.cc:1918] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:38:36.262089 2807 log.cc:1196] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/9829f1c14b36473593001ece1cb1e390
I20260810 14:38:36.262264 2805 log.cc:1196] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/9829f1c14b36473593001ece1cb1e390
I20260810 14:38:36.262739 2802 ts_tablet_manager.cc:1905] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:36.263156 2805 ts_tablet_manager.cc:1939] T 9829f1c14b36473593001ece1cb1e390 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting consensus metadata
I20260810 14:38:36.264529 1771 catalog_manager.cc:4841] TS 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387): tablet 9829f1c14b36473593001ece1cb1e390 (table test-table [id=47e26288648c4ca893599501f33347f9]) successfully deleted
I20260810 14:38:36.267066 2807 ts_tablet_manager.cc:1939] T 9829f1c14b36473593001ece1cb1e390 P 04da3915c6bf4d5f9cd132f484029a80: Deleting consensus metadata
I20260810 14:38:36.268409 1770 catalog_manager.cc:4841] TS 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573): tablet 9829f1c14b36473593001ece1cb1e390 (table test-table [id=47e26288648c4ca893599501f33347f9]) successfully deleted
I20260810 14:38:36.269194 2802 ts_tablet_manager.cc:1918] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:38:36.269346 2802 log.cc:1196] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/9829f1c14b36473593001ece1cb1e390
I20260810 14:38:36.269758 2802 ts_tablet_manager.cc:1939] T 9829f1c14b36473593001ece1cb1e390 P 894207e400bc4b91ad71973a424b3c49: Deleting consensus metadata
I20260810 14:38:36.270704 1768 catalog_manager.cc:4841] TS 894207e400bc4b91ad71973a424b3c49 (127.21.55.66:41563): tablet 9829f1c14b36473593001ece1cb1e390 (table test-table [id=47e26288648c4ca893599501f33347f9]) successfully deleted
Aug 10 14:38:36 dist-test-slave-h6n0 krb5kdc[1721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:38:36 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:36 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:38:36.328647 2824 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:36.311465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.21.55.126:43481)
0810 14:38:36.316173 (+ 4708us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:36.316178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:36.316196 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:38:36.316256 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:36.316260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:36.316364 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:36.316472 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:36.318376 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.319295 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.320989 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.321382 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.321477 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:36.328343 (+ 6866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47088: BlockingRecv error: recv got EOF from 127.0.0.1:47088 (error 108)
Metrics: {"server-negotiator.queue_time_us":4596,"thread_start_us":4441,"threads_started":1}
Aug 10 14:38:36 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM
I20260810 14:38:36.351023 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:36.331757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.21.55.126:43481)
0810 14:38:36.331953 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:36.331959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:36.332351 (+ 392us) server_negotiation.cc:407] Connection header received
0810 14:38:36.332475 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:36.332481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:36.332561 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:36.332670 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:36.333884 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.335055 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.336451 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.337526 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.337828 (+ 302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:36.342444 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:36.342452 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:36.342457 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:36.342521 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:36.346596 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:36.347822 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:36.347828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:36.347832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:36.348080 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:36.348833 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:36.348841 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:36.348844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:36.349340 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:36.349645 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:36.349958 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 14:38:36.350645 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:38:36.355674 1796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:38:36.356329 1796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:38:36.364259 2034 tablet_service.cc:1467] Processing CreateTablet for tablet 8dff855040d74983a6c31d9cae115876 (DEFAULT_TABLE table=test-table [id=9b76434cf75941188027130cdffadb53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:36.364856 2034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dff855040d74983a6c31d9cae115876. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:36.364727 2526 tablet_service.cc:1467] Processing CreateTablet for tablet 8dff855040d74983a6c31d9cae115876 (DEFAULT_TABLE table=test-table [id=9b76434cf75941188027130cdffadb53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:36.365243 2526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dff855040d74983a6c31d9cae115876. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:36.370718 2272 tablet_service.cc:1467] Processing CreateTablet for tablet 8dff855040d74983a6c31d9cae115876 (DEFAULT_TABLE table=test-table [id=9b76434cf75941188027130cdffadb53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:36.371357 2272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dff855040d74983a6c31d9cae115876. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:36.377071 2835 tablet_bootstrap.cc:492] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80: Bootstrap starting.
I20260810 14:38:36.377362 2834 tablet_bootstrap.cc:492] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Bootstrap starting.
I20260810 14:38:36.379231 2834 tablet_bootstrap.cc:654] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:36.379269 2835 tablet_bootstrap.cc:654] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:36.386669 2840 tablet_bootstrap.cc:492] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49: Bootstrap starting.
I20260810 14:38:36.388161 2834 tablet_bootstrap.cc:492] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7: No bootstrap required, opened a new log
I20260810 14:38:36.388379 2834 ts_tablet_manager.cc:1397] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260810 14:38:36.389381 2834 raft_consensus.cc:348] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.389725 2834 raft_consensus.cc:374] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:36.389832 2834 raft_consensus.cc:729] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac0b4c39b9449e09c94c9c4c8b838a7, State: Initialized, Role: FOLLOWER
I20260810 14:38:36.390136 2834 consensus_queue.cc:260] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.390965 2834 ts_tablet_manager.cc:1428] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 14:38:36.391192 2840 tablet_bootstrap.cc:654] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:36.396799 2835 tablet_bootstrap.cc:492] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80: No bootstrap required, opened a new log
I20260810 14:38:36.396952 2835 ts_tablet_manager.cc:1397] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.009s
I20260810 14:38:36.397615 2835 raft_consensus.cc:348] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.397938 2835 raft_consensus.cc:374] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:36.398082 2835 raft_consensus.cc:729] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04da3915c6bf4d5f9cd132f484029a80, State: Initialized, Role: FOLLOWER
I20260810 14:38:36.398343 2840 tablet_bootstrap.cc:492] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49: No bootstrap required, opened a new log
I20260810 14:38:36.398509 2840 ts_tablet_manager.cc:1397] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260810 14:38:36.398429 2835 consensus_queue.cc:260] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.399197 2835 ts_tablet_manager.cc:1428] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:38:36.399374 2840 raft_consensus.cc:348] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.399634 2840 raft_consensus.cc:374] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:36.399730 2840 raft_consensus.cc:729] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894207e400bc4b91ad71973a424b3c49, State: Initialized, Role: FOLLOWER
I20260810 14:38:36.399982 2840 consensus_queue.cc:260] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.400569 2840 ts_tablet_manager.cc:1428] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 14:38:36.402282 2790 raft_consensus.cc:482] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:38:36.402451 2790 raft_consensus.cc:504] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.403218 2790 leader_election.cc:288] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573), 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387)
I20260810 14:38:36.403928 2558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dff855040d74983a6c31d9cae115876" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" is_pre_election: true
I20260810 14:38:36.404206 2077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dff855040d74983a6c31d9cae115876" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04da3915c6bf4d5f9cd132f484029a80" is_pre_election: true
I20260810 14:38:36.404270 2558 raft_consensus.cc:2454] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 0.
I20260810 14:38:36.404454 2077 raft_consensus.cc:2454] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 0.
I20260810 14:38:36.404909 2188 leader_election.cc:302] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04da3915c6bf4d5f9cd132f484029a80, 894207e400bc4b91ad71973a424b3c49; no voters:
I20260810 14:38:36.405256 2790 raft_consensus.cc:2781] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:38:36.405387 2790 raft_consensus.cc:482] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:38:36.405483 2790 raft_consensus.cc:3037] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:36.407666 2790 raft_consensus.cc:504] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.408236 2790 leader_election.cc:288] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 election: Requested vote from peers 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573), 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387)
I20260810 14:38:36.408712 2077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dff855040d74983a6c31d9cae115876" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04da3915c6bf4d5f9cd132f484029a80"
I20260810 14:38:36.408728 2558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dff855040d74983a6c31d9cae115876" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7"
I20260810 14:38:36.408939 2077 raft_consensus.cc:3037] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:36.408952 2558 raft_consensus.cc:3037] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:36.411404 2077 raft_consensus.cc:2454] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 1.
I20260810 14:38:36.411837 2188 leader_election.cc:302] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04da3915c6bf4d5f9cd132f484029a80, 894207e400bc4b91ad71973a424b3c49; no voters:
I20260810 14:38:36.412096 2558 raft_consensus.cc:2454] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 1.
I20260810 14:38:36.412153 2790 raft_consensus.cc:2781] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:36.412524 2790 raft_consensus.cc:686] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 1 LEADER]: Becoming Leader. State: Replica: 894207e400bc4b91ad71973a424b3c49, State: Running, Role: LEADER
I20260810 14:38:36.412801 2790 consensus_queue.cc:237] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.416150 1796 catalog_manager.cc:5495] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 894207e400bc4b91ad71973a424b3c49 (127.21.55.66). New cstate: current_term: 1 leader_uuid: "894207e400bc4b91ad71973a424b3c49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } health_report { overall_health: UNKNOWN } } }
I20260810 14:38:36.441859 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:36.434748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42178 (local address 127.21.55.66:41563)
0810 14:38:36.435013 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:36.435019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:36.435042 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:38:36.435198 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:36.435204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:36.435298 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:36.435406 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:38:36.436258 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.437432 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.438913 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.439638 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.439802 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:36.440656 (+ 854us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:38:36.440851 (+ 195us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:38:36.440962 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:36.441432 (+ 470us) server_negotiation.cc:299] Negotiation successful
0810 14:38:36.441529 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 14:38:36.445885 2077 raft_consensus.cc:1261] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Refusing update from remote peer 894207e400bc4b91ad71973a424b3c49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:36.446048 2558 raft_consensus.cc:1261] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Refusing update from remote peer 894207e400bc4b91ad71973a424b3c49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:36.446826 2698 consensus_queue.cc:1035] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:36.447139 2790 consensus_queue.cc:1035] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:36.461274 1796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:36.461573 1796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:36.465551 1796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6: Sending DeleteTablet for 3 replicas of tablet 8dff855040d74983a6c31d9cae115876
I20260810 14:38:36.466468 2526 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dff855040d74983a6c31d9cae115876 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:36 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:52044
I20260810 14:38:36.466775 2034 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dff855040d74983a6c31d9cae115876 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:36 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:54704
I20260810 14:38:36.467070 2807 tablet_replica.cc:330] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80: stopping tablet replica
I20260810 14:38:36.467069 2805 tablet_replica.cc:330] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7: stopping tablet replica
I20260810 14:38:36.467280 2805 raft_consensus.cc:2229] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:36.467301 2807 raft_consensus.cc:2229] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:36.467514 2805 raft_consensus.cc:2258] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:36.467525 2807 raft_consensus.cc:2258] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:36.468747 2805 ts_tablet_manager.cc:1905] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:36.468907 2807 ts_tablet_manager.cc:1905] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:36.474973 2272 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dff855040d74983a6c31d9cae115876 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:36 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:42172
I20260810 14:38:36.475301 2802 tablet_replica.cc:330] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49: stopping tablet replica
I20260810 14:38:36.475508 2802 raft_consensus.cc:2229] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:38:36.475872 2802 raft_consensus.cc:2258] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:36.476044 2526 tablet_service.cc:1514] Processing DeleteTablet for tablet 8dff855040d74983a6c31d9cae115876 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:36 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:52044
I20260810 14:38:36.477154 2807 ts_tablet_manager.cc:1918] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:36.477435 2807 log.cc:1196] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/8dff855040d74983a6c31d9cae115876
I20260810 14:38:36.477444 2802 ts_tablet_manager.cc:1905] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:36.477181 1796 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 14:38:36.478060 2807 ts_tablet_manager.cc:1939] T 8dff855040d74983a6c31d9cae115876 P 04da3915c6bf4d5f9cd132f484029a80: Deleting consensus metadata
W20260810 14:38:36.478492 1796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:38:36.480919 1770 catalog_manager.cc:4841] TS 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573): tablet 8dff855040d74983a6c31d9cae115876 (table test-table [id=9b76434cf75941188027130cdffadb53]) successfully deleted
I20260810 14:38:36.483328 2805 ts_tablet_manager.cc:1918] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:36.483548 2805 log.cc:1196] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/8dff855040d74983a6c31d9cae115876
I20260810 14:38:36.484097 2805 ts_tablet_manager.cc:1939] T 8dff855040d74983a6c31d9cae115876 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting consensus metadata
I20260810 14:38:36.484311 2802 ts_tablet_manager.cc:1918] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:38:36.484475 2802 log.cc:1196] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/8dff855040d74983a6c31d9cae115876
I20260810 14:38:36.485041 2802 ts_tablet_manager.cc:1939] T 8dff855040d74983a6c31d9cae115876 P 894207e400bc4b91ad71973a424b3c49: Deleting consensus metadata
I20260810 14:38:36.485689 1771 catalog_manager.cc:4841] TS 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387): tablet 8dff855040d74983a6c31d9cae115876 (table test-table [id=9b76434cf75941188027130cdffadb53]) successfully deleted
W20260810 14:38:36.486282 1771 catalog_manager.cc:4808] TS 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387): delete failed for tablet 8dff855040d74983a6c31d9cae115876 because the tablet was not found. No further retry: Not found: Tablet not found: 8dff855040d74983a6c31d9cae115876
I20260810 14:38:36.487427 1768 catalog_manager.cc:4841] TS 894207e400bc4b91ad71973a424b3c49 (127.21.55.66:41563): tablet 8dff855040d74983a6c31d9cae115876 (table test-table [id=9b76434cf75941188027130cdffadb53]) successfully deleted
I20260810 14:38:36.491354 2034 tablet_service.cc:1467] Processing CreateTablet for tablet b4bbe7ed18684fd7b6eb0f3bd9a96522 (DEFAULT_TABLE table=test-table [id=9d564b9daab64c71b4a63061b9bf831c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:36.491679 2526 tablet_service.cc:1467] Processing CreateTablet for tablet b4bbe7ed18684fd7b6eb0f3bd9a96522 (DEFAULT_TABLE table=test-table [id=9d564b9daab64c71b4a63061b9bf831c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:36.491914 2034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4bbe7ed18684fd7b6eb0f3bd9a96522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:36.492115 2526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4bbe7ed18684fd7b6eb0f3bd9a96522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:36.491714 2272 tablet_service.cc:1467] Processing CreateTablet for tablet b4bbe7ed18684fd7b6eb0f3bd9a96522 (DEFAULT_TABLE table=test-table [id=9d564b9daab64c71b4a63061b9bf831c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:38:36.492825 2272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4bbe7ed18684fd7b6eb0f3bd9a96522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:36.499217 2835 tablet_bootstrap.cc:492] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80: Bootstrap starting.
I20260810 14:38:36.499835 2834 tablet_bootstrap.cc:492] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Bootstrap starting.
I20260810 14:38:36.500881 2835 tablet_bootstrap.cc:654] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:36.501591 2834 tablet_bootstrap.cc:654] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:36.504477 2834 tablet_bootstrap.cc:492] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7: No bootstrap required, opened a new log
I20260810 14:38:36.504788 2834 ts_tablet_manager.cc:1397] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 14:38:36.504926 2840 tablet_bootstrap.cc:492] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49: Bootstrap starting.
I20260810 14:38:36.505798 2834 raft_consensus.cc:348] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.506664 2834 raft_consensus.cc:374] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:36.506780 2834 raft_consensus.cc:729] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ac0b4c39b9449e09c94c9c4c8b838a7, State: Initialized, Role: FOLLOWER
I20260810 14:38:36.506906 2840 tablet_bootstrap.cc:654] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:36.507727 2835 tablet_bootstrap.cc:492] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80: No bootstrap required, opened a new log
I20260810 14:38:36.507963 2835 ts_tablet_manager.cc:1397] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260810 14:38:36.507112 2834 consensus_queue.cc:260] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.509042 2834 ts_tablet_manager.cc:1428] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 14:38:36.511301 2835 raft_consensus.cc:348] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.511940 2835 raft_consensus.cc:374] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:36.512146 2835 raft_consensus.cc:729] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04da3915c6bf4d5f9cd132f484029a80, State: Initialized, Role: FOLLOWER
I20260810 14:38:36.512670 2835 consensus_queue.cc:260] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.514269 2835 ts_tablet_manager.cc:1428] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260810 14:38:36.516904 2840 tablet_bootstrap.cc:492] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49: No bootstrap required, opened a new log
I20260810 14:38:36.517177 2840 ts_tablet_manager.cc:1397] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260810 14:38:36.518489 2840 raft_consensus.cc:348] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.518805 2840 raft_consensus.cc:374] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:36.519181 2840 raft_consensus.cc:729] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894207e400bc4b91ad71973a424b3c49, State: Initialized, Role: FOLLOWER
I20260810 14:38:36.519609 2840 consensus_queue.cc:260] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.520805 2840 ts_tablet_manager.cc:1428] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:38:36.637197 2853 raft_consensus.cc:482] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:38:36.637431 2853 raft_consensus.cc:504] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.638413 2853 leader_election.cc:288] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573), 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387)
I20260810 14:38:36.639276 2077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4bbe7ed18684fd7b6eb0f3bd9a96522" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04da3915c6bf4d5f9cd132f484029a80" is_pre_election: true
I20260810 14:38:36.639446 2558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4bbe7ed18684fd7b6eb0f3bd9a96522" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" is_pre_election: true
I20260810 14:38:36.639602 2077 raft_consensus.cc:2454] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 0.
I20260810 14:38:36.639762 2558 raft_consensus.cc:2454] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 0.
I20260810 14:38:36.640151 2188 leader_election.cc:302] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04da3915c6bf4d5f9cd132f484029a80, 894207e400bc4b91ad71973a424b3c49; no voters:
I20260810 14:38:36.640445 2853 raft_consensus.cc:2781] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:38:36.640583 2853 raft_consensus.cc:482] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:38:36.640693 2853 raft_consensus.cc:3037] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:36.643144 2853 raft_consensus.cc:504] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.643774 2853 leader_election.cc:288] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 election: Requested vote from peers 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573), 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387)
I20260810 14:38:36.644279 2077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4bbe7ed18684fd7b6eb0f3bd9a96522" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04da3915c6bf4d5f9cd132f484029a80"
I20260810 14:38:36.644353 2558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4bbe7ed18684fd7b6eb0f3bd9a96522" candidate_uuid: "894207e400bc4b91ad71973a424b3c49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7"
I20260810 14:38:36.644536 2077 raft_consensus.cc:3037] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:36.644626 2558 raft_consensus.cc:3037] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:36.646611 2077 raft_consensus.cc:2454] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 1.
I20260810 14:38:36.647066 2188 leader_election.cc:302] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04da3915c6bf4d5f9cd132f484029a80, 894207e400bc4b91ad71973a424b3c49; no voters:
I20260810 14:38:36.647336 2853 raft_consensus.cc:2781] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:36.647506 2853 raft_consensus.cc:686] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 1 LEADER]: Becoming Leader. State: Replica: 894207e400bc4b91ad71973a424b3c49, State: Running, Role: LEADER
I20260810 14:38:36.647770 2853 consensus_queue.cc:237] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } }
I20260810 14:38:36.648211 2558 raft_consensus.cc:2454] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894207e400bc4b91ad71973a424b3c49 in term 1.
I20260810 14:38:36.650836 1796 catalog_manager.cc:5495] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 894207e400bc4b91ad71973a424b3c49 (127.21.55.66). New cstate: current_term: 1 leader_uuid: "894207e400bc4b91ad71973a424b3c49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "894207e400bc4b91ad71973a424b3c49" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 41563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 } health_report { overall_health: UNKNOWN } } }
I20260810 14:38:36.699188 2558 raft_consensus.cc:1261] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Refusing update from remote peer 894207e400bc4b91ad71973a424b3c49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:36.699481 2077 raft_consensus.cc:1261] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Refusing update from remote peer 894207e400bc4b91ad71973a424b3c49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:38:36.700007 2698 consensus_queue.cc:1035] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ac0b4c39b9449e09c94c9c4c8b838a7" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:36.700490 2853 consensus_queue.cc:1035] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04da3915c6bf4d5f9cd132f484029a80" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:38:36.705049 2764 mvcc.cc:205] Tried to move back new op lower bound from 7316982647594627072 to 7316982647392116736. Current Snapshot: MvccSnapshot[applied={T|T < 7316982647594627072}]
I20260810 14:38:36.705456 2765 mvcc.cc:205] Tried to move back new op lower bound from 7316982647594627072 to 7316982647392116736. Current Snapshot: MvccSnapshot[applied={T|T < 7316982647594627072}]
I20260810 14:38:36.758467 1792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:36.758769 1792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:38:36.761848 1792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08bd303162484c20bcfee8de64a443c6: Sending DeleteTablet for 3 replicas of tablet b4bbe7ed18684fd7b6eb0f3bd9a96522
I20260810 14:38:36.762687 2272 tablet_service.cc:1514] Processing DeleteTablet for tablet b4bbe7ed18684fd7b6eb0f3bd9a96522 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:36 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:42172
I20260810 14:38:36.762765 2526 tablet_service.cc:1514] Processing DeleteTablet for tablet b4bbe7ed18684fd7b6eb0f3bd9a96522 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:36 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:52044
I20260810 14:38:36.762862 2034 tablet_service.cc:1514] Processing DeleteTablet for tablet b4bbe7ed18684fd7b6eb0f3bd9a96522 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:38:36 UTC) from {username='oryx', principal='oryx/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:54704
I20260810 14:38:36.763006 2802 tablet_replica.cc:330] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49: stopping tablet replica
I20260810 14:38:36.763196 2802 raft_consensus.cc:2229] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:38:36.763203 2807 tablet_replica.cc:330] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80: stopping tablet replica
I20260810 14:38:36.763403 2807 raft_consensus.cc:2229] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:36.763582 2802 raft_consensus.cc:2258] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:36.763633 2807 raft_consensus.cc:2258] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:36.763929 2805 tablet_replica.cc:330] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7: stopping tablet replica
I20260810 14:38:36.764127 2805 raft_consensus.cc:2229] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:38:36.764375 2805 raft_consensus.cc:2258] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:38:36.765431 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 1922
I20260810 14:38:36.765723 2807 ts_tablet_manager.cc:1905] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:36.771070 2805 ts_tablet_manager.cc:1905] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:36.771570 2802 ts_tablet_manager.cc:1905] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:38:36.775321 2807 ts_tablet_manager.cc:1918] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:38:36.775528 2807 log.cc:1196] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/b4bbe7ed18684fd7b6eb0f3bd9a96522
I20260810 14:38:36.776103 2807 ts_tablet_manager.cc:1939] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 04da3915c6bf4d5f9cd132f484029a80: Deleting consensus metadata
I20260810 14:38:36.776196 2805 ts_tablet_manager.cc:1918] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:38:36.776338 2805 log.cc:1196] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/b4bbe7ed18684fd7b6eb0f3bd9a96522
I20260810 14:38:36.776830 2805 ts_tablet_manager.cc:1939] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 0ac0b4c39b9449e09c94c9c4c8b838a7: Deleting consensus metadata
I20260810 14:38:36.777766 1770 catalog_manager.cc:4841] TS 04da3915c6bf4d5f9cd132f484029a80 (127.21.55.65:40573): tablet b4bbe7ed18684fd7b6eb0f3bd9a96522 (table test-table [id=9d564b9daab64c71b4a63061b9bf831c]) successfully deleted
I20260810 14:38:36.778062 1771 catalog_manager.cc:4841] TS 0ac0b4c39b9449e09c94c9c4c8b838a7 (127.21.55.67:41387): tablet b4bbe7ed18684fd7b6eb0f3bd9a96522 (table test-table [id=9d564b9daab64c71b4a63061b9bf831c]) successfully deleted
I20260810 14:38:36.780719 2802 ts_tablet_manager.cc:1918] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:38:36.780834 2802 log.cc:1196] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/b4bbe7ed18684fd7b6eb0f3bd9a96522
I20260810 14:38:36.781348 2802 ts_tablet_manager.cc:1939] T b4bbe7ed18684fd7b6eb0f3bd9a96522 P 894207e400bc4b91ad71973a424b3c49: Deleting consensus metadata
I20260810 14:38:36.782291 1768 catalog_manager.cc:4841] TS 894207e400bc4b91ad71973a424b3c49 (127.21.55.66:41563): tablet b4bbe7ed18684fd7b6eb0f3bd9a96522 (table test-table [id=9d564b9daab64c71b4a63061b9bf831c]) successfully deleted
Aug 10 14:38:36 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.55.65@KRBTEST.COM
I20260810 14:38:36.796108 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:36.766362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54732 (local address 127.21.55.65:40573)
0810 14:38:36.766923 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:36.766932 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:36.778709 (+ 11777us) server_negotiation.cc:407] Connection header received
0810 14:38:36.779221 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:36.779227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:36.779350 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:36.779479 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:36.781201 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.782060 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.783549 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.784012 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.784174 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:36.789111 (+ 4937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:36.789126 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:36.789142 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:36.789173 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:36.792653 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:36.793549 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:36.793558 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:36.793561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:36.793751 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:36.794421 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:36.794425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:36.794428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:36.795001 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:36.795284 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:36.795606 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 14:38:36.795819 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":292,"threads_started":1}
I20260810 14:38:36.796875 2145 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:36.930472 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 1922
I20260810 14:38:36.965801 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 2160
Aug 10 14:38:36 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.55.66@KRBTEST.COM
I20260810 14:38:36.989652 2872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:36.966822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42192 (local address 127.21.55.66:41563)
0810 14:38:36.967277 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:36.967282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:36.967299 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:38:36.967357 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:36.967361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:36.967437 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:36.967539 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:36.968813 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.969812 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.971500 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:36.972085 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:36.972205 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:36.982073 (+ 9868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:36.982089 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:36.982105 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:36.982142 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:36.985656 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:36.986389 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:36.986396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:36.986398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:36.986635 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:36.987506 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:36.987512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:36.987514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:36.988100 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:36.988582 (+ 482us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:36.988888 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 14:38:36.989081 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":246,"threads_started":1}
I20260810 14:38:36.990749 2390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:37.128016 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 2160
I20260810 14:38:37.163154 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 2408
Aug 10 14:38:37 dist-test-slave-h6n0 krb5kdc[1721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.55.67@KRBTEST.COM
I20260810 14:38:37.183949 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:37.164060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52058 (local address 127.21.55.67:41387)
0810 14:38:37.164630 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:37.164635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:37.164653 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:38:37.164711 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:37.164714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:37.164782 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:37.164900 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:37.166356 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:37.167496 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:37.169130 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:37.169667 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:37.169827 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:37.175635 (+ 5808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:37.175663 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:37.175685 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:37.175745 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:37.179683 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:37.180978 (+ 1295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:37.180990 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:37.180993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:37.181272 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:37.181954 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:37.181960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:37.181963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:37.182484 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:37.182870 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:37.183485 (+ 615us) server_negotiation.cc:299] Negotiation successful
0810 14:38:37.183684 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":281,"threads_started":1}
I20260810 14:38:37.184785 2630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:37.288311 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 2408
I20260810 14:38:37.314504 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 1741
I20260810 14:38:37.315956 1867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:37.434787 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 1741
2026-08-10T14:38:37Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7579 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:38:37 dist-test-slave-h6n0 krb5kdc[2883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:38:37 dist-test-slave-h6n0 krb5kdc[2883](info): set up 2 sockets
Aug 10 14:38:37 dist-test-slave-h6n0 krb5kdc[2883](info): commencing operation
krb5kdc: starting...
W20260810 14:38:38.848249 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.351s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:38:39 dist-test-slave-h6n0 krb5kdc[2883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:38:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:38:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:38:39.142885 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:40307
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:41593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:40307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:39.488863 2899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:39.489156 2899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:39.489279 2899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:39.499140 2899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:39.499323 2899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:39.499454 2899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:39.499575 2899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:39.499660 2899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:38:39.512394 2899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:40307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:40307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:39.514078 2899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:39.515587 2899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:39.529415 2910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:39.529505 2911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:39.529459 2914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:39.535495 2899 server_base.cc:1034] running on GCE node
I20260810 14:38:39.536331 2899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:39.537773 2899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:39.539004 2899 hybrid_clock.cc:648] HybridClock initialized: now 1786372719538915 us; error 65 us; skew 500 ppm
I20260810 14:38:39.539417 2899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T14:38:39Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2165 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:38:39 dist-test-slave-h6n0 krb5kdc[2922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:38:39 dist-test-slave-h6n0 krb5kdc[2922](info): set up 2 sockets
Aug 10 14:38:39 dist-test-slave-h6n0 krb5kdc[2922](info): commencing operation
krb5kdc: starting...
W20260810 14:38:41.003290 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:38:41 dist-test-slave-h6n0 krb5kdc[2922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:38:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:38:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:38:41.287698 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:33539
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:33539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:41.626130 2942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:41.626466 2942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:41.626657 2942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:41.633970 2942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:41.634341 2942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:41.634476 2942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:41.634712 2942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:41.634861 2942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:38:41.646093 2942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:33539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:33539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:41.647524 2942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:41.648870 2942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:41.662226 2954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:41.662279 2957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:41.662699 2952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:41.663188 2942 server_base.cc:1034] running on GCE node
I20260810 14:38:41.663978 2942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:41.665351 2942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:41.666654 2942 hybrid_clock.cc:648] HybridClock initialized: now 1786372721666582 us; error 67 us; skew 500 ppm
I20260810 14:38:41.667053 2942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:41 dist-test-slave-h6n0 krb5kdc[2922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372721, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:41.671083 2942 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:38:41.672703 2942 webserver.cc:460] Webserver started at http://127.21.55.126:45317/ using document root <none> and password file <none>
I20260810 14:38:41.673421 2942 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:41.673575 2942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:41.673990 2942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:41.677006 2942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "5425face473c401bb2421dccabe167df"
format_stamp: "Formatted at 2026-08-10 14:38:41 on dist-test-slave-h6n0"
server_key: "129f4ab3e3df1191136cb870098a0438"
server_key_iv: "39bd6abe5fe3a8f3bab36c8250b10fca"
server_key_version: "encryptionkey@0"
I20260810 14:38:41.677789 2942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "5425face473c401bb2421dccabe167df"
format_stamp: "Formatted at 2026-08-10 14:38:41 on dist-test-slave-h6n0"
server_key: "129f4ab3e3df1191136cb870098a0438"
server_key_iv: "39bd6abe5fe3a8f3bab36c8250b10fca"
server_key_version: "encryptionkey@0"
I20260810 14:38:41.683018 2942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 14:38:41.686429 2966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:41.688022 2942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:38:41.688246 2942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "5425face473c401bb2421dccabe167df"
format_stamp: "Formatted at 2026-08-10 14:38:41 on dist-test-slave-h6n0"
server_key: "129f4ab3e3df1191136cb870098a0438"
server_key_iv: "39bd6abe5fe3a8f3bab36c8250b10fca"
server_key_version: "encryptionkey@0"
I20260810 14:38:41.688746 2942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:41.713191 2942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:41.718842 2942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:41.719202 2942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:41.741673 2942 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:33539
I20260810 14:38:41.741695 3057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:33539 every 8 connection(s)
I20260810 14:38:41.743141 2942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:38:41.748221 3058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:38:41.753695 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 2942
I20260810 14:38:41.753942 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:38:41.754962 21725 external_mini_cluster.cc:1428] Setting key 38b56099c9f53bbb3946925a23a02e12
I20260810 14:38:41.761415 3058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df: Bootstrap starting.
I20260810 14:38:41.764753 3058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df: Neither blocks nor log segments found. Creating new log.
I20260810 14:38:41.766185 3058 log.cc:824] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:41.769840 3058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df: No bootstrap required, opened a new log
Aug 10 14:38:41 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:41.779188 3065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:41.757273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35108 (local address 127.21.55.126:33539)
0810 14:38:41.758179 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:41.758191 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:41.758235 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:38:41.760058 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:41.760080 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:41.760441 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:41.760810 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:41.762387 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:41.763763 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:41.765052 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:41.766183 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:41.766404 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:41.771558 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:41.771565 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:41.771579 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:41.771612 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:41.775078 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:41.775944 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:41.775950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:41.775956 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:41.776114 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:41.776556 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:41.776559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:41.776561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:41.777040 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:41.777251 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:41.777736 (+ 485us) server_negotiation.cc:299] Negotiation successful
0810 14:38:41.778092 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":455,"threads_started":1}
I20260810 14:38:41.779961 3058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } }
I20260810 14:38:41.780323 3058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:38:41.780450 3058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5425face473c401bb2421dccabe167df, State: Initialized, Role: FOLLOWER
I20260810 14:38:41.781113 3058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } }
I20260810 14:38:41.781409 3058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:38:41.781553 3058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:38:41.781733 3058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:38:41.783913 3058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } }
I20260810 14:38:41.784395 3058 leader_election.cc:302] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5425face473c401bb2421dccabe167df; no voters:
I20260810 14:38:41.784925 3058 leader_election.cc:288] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:38:41.785068 3069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:38:41.786039 3069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 1 LEADER]: Becoming Leader. State: Replica: 5425face473c401bb2421dccabe167df, State: Running, Role: LEADER
I20260810 14:38:41.787081 3058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:41.787027 3069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } }
I20260810 14:38:41.791493 3071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5425face473c401bb2421dccabe167df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } } }
I20260810 14:38:41.791877 3071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:41.792544 3072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5425face473c401bb2421dccabe167df. Latest consensus state: current_term: 1 leader_uuid: "5425face473c401bb2421dccabe167df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } } }
I20260810 14:38:41.792878 3072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:41.794912 3087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:41.800829 3087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:41.809684 3087 catalog_manager.cc:1347] Generated new cluster ID: 435fccfb08c54f1ebab571177bfcf313
I20260810 14:38:41.809801 3087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:41.834970 3087 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:38:41.836277 3087 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:41.844776 3087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df: Generated new TSK 0
I20260810 14:38:41.845980 3087 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:38:41.925933 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33539
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:42.289042 3103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:42.289328 3103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:42.289425 3103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:42.297263 3103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:42.297374 3103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:42.297492 3103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:38:42.307169 3103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:42.308449 3103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:42.309707 3103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:42.324401 3110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:42.324481 3111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:42.324443 3113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:42.325251 3103 server_base.cc:1034] running on GCE node
I20260810 14:38:42.326232 3103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:42.327448 3103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:42.328723 3103 hybrid_clock.cc:648] HybridClock initialized: now 1786372722328662 us; error 44 us; skew 500 ppm
I20260810 14:38:42.329393 3103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:42 dist-test-slave-h6n0 krb5kdc[2922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372722, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:42.337724 3103 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:38:42.339551 3103 webserver.cc:460] Webserver started at http://127.21.55.65:39171/ using document root <none> and password file <none>
I20260810 14:38:42.340277 3103 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:42.340443 3103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:42.340773 3103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:42.345019 3103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "9215509df28d4acc8216b9345c99d483"
format_stamp: "Formatted at 2026-08-10 14:38:42 on dist-test-slave-h6n0"
server_key: "0ecb6e423683b5c7e1eddb802125e7df"
server_key_iv: "7851c078ed8953ba1d80bbd805daaddb"
server_key_version: "encryptionkey@0"
I20260810 14:38:42.346207 3103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "9215509df28d4acc8216b9345c99d483"
format_stamp: "Formatted at 2026-08-10 14:38:42 on dist-test-slave-h6n0"
server_key: "0ecb6e423683b5c7e1eddb802125e7df"
server_key_iv: "7851c078ed8953ba1d80bbd805daaddb"
server_key_version: "encryptionkey@0"
I20260810 14:38:42.354137 3103 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260810 14:38:42.358739 3128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:42.360610 3103 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:38:42.360949 3103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "9215509df28d4acc8216b9345c99d483"
format_stamp: "Formatted at 2026-08-10 14:38:42 on dist-test-slave-h6n0"
server_key: "0ecb6e423683b5c7e1eddb802125e7df"
server_key_iv: "7851c078ed8953ba1d80bbd805daaddb"
server_key_version: "encryptionkey@0"
I20260810 14:38:42.361408 3103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:42.382818 3103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:42.389108 3103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:42.389523 3103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:42.391022 3103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:42.392992 3103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:42.393114 3103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:42.393294 3103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:42.393378 3103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:42.435411 3103 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:37277
I20260810 14:38:42.435454 3307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:37277 every 8 connection(s)
I20260810 14:38:42.436827 3103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:38:42.447337 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 3103
I20260810 14:38:42.447608 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:38:42.448356 21725 external_mini_cluster.cc:1428] Setting key 24e144681ca99fedcbc7f1aa0b0fcdf5
Aug 10 14:38:42 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372722, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:42.463501 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:42.439822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:35611 (local address 127.21.55.126:33539)
0810 14:38:42.440478 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:42.440517 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:42.441174 (+ 657us) server_negotiation.cc:407] Connection header received
0810 14:38:42.443281 (+ 2107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:42.443293 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:42.443400 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:42.443522 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:42.446109 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:42.447188 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:42.449792 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:42.450466 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:42.450785 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:42.454513 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:42.454522 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:42.454561 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:42.454608 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:42.459065 (+ 4457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:42.459923 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:42.459931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:42.459937 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:42.460108 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:42.460602 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:42.460609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:42.460615 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:42.461018 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:42.461174 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:42.462695 (+ 1521us) server_negotiation.cc:299] Negotiation successful
0810 14:38:42.463034 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":392,"threads_started":1}
I20260810 14:38:42.464896 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:42.440031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:35611)
0810 14:38:42.440931 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:42.440989 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:42.442871 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:42.444009 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:42.444023 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:42.444850 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:42.445639 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:42.445664 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:42.447431 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:42.447437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:42.449511 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:42.449561 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:42.450896 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:42.450912 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:42.451143 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:42.451912 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:42.451941 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:42.454254 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:42.459251 (+ 4997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:42.459261 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:42.459275 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:42.459765 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:42.460258 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:42.460262 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:42.460265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:42.460481 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:42.461388 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:42.461404 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:42.462090 (+ 686us) client_negotiation.cc:769] Sending connection context
0810 14:38:42.462513 (+ 423us) client_negotiation.cc:240] Negotiation successful
0810 14:38:42.463324 (+ 811us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":417,"threads_started":1}
I20260810 14:38:42.467532 3309 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33539
I20260810 14:38:42.468000 3309 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:42.468900 3309 heartbeater.cc:511] Master 127.21.55.126:33539 requested a full tablet report, sending...
I20260810 14:38:42.471493 2994 ts_manager.cc:194] Registered new tserver with Master: 9215509df28d4acc8216b9345c99d483 (127.21.55.65:37277)
I20260810 14:38:42.473023 2994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:35611
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:38:42.518790 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33539
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:42.877269 3319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:42.877560 3319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:42.877657 3319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:42.885339 3319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:42.885475 3319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:42.885598 3319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:38:42.896049 3319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:42.897389 3319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:42.898815 3319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:42.914921 3327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:42.915014 3331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:42.914887 3328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:42.920267 3319 server_base.cc:1034] running on GCE node
I20260810 14:38:42.920991 3319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:42.922092 3319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:42.923507 3319 hybrid_clock.cc:648] HybridClock initialized: now 1786372722923413 us; error 43 us; skew 500 ppm
I20260810 14:38:42.924088 3319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:42 dist-test-slave-h6n0 krb5kdc[2922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372722, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:42.928781 3319 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:38:42.931056 3319 webserver.cc:460] Webserver started at http://127.21.55.66:45373/ using document root <none> and password file <none>
I20260810 14:38:42.931942 3319 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:42.932121 3319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:42.932503 3319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:42.935874 3319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "16967801d4b149b29f2ee7d1fd58e7e5"
format_stamp: "Formatted at 2026-08-10 14:38:42 on dist-test-slave-h6n0"
server_key: "29b22d4f6dfc94d61c795abd67993bbc"
server_key_iv: "f633224303864f52c52025a1a1bf6faf"
server_key_version: "encryptionkey@0"
I20260810 14:38:42.936764 3319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "16967801d4b149b29f2ee7d1fd58e7e5"
format_stamp: "Formatted at 2026-08-10 14:38:42 on dist-test-slave-h6n0"
server_key: "29b22d4f6dfc94d61c795abd67993bbc"
server_key_iv: "f633224303864f52c52025a1a1bf6faf"
server_key_version: "encryptionkey@0"
I20260810 14:38:42.942649 3319 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 14:38:42.947091 3344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:42.949008 3319 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:38:42.949211 3319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "16967801d4b149b29f2ee7d1fd58e7e5"
format_stamp: "Formatted at 2026-08-10 14:38:42 on dist-test-slave-h6n0"
server_key: "29b22d4f6dfc94d61c795abd67993bbc"
server_key_iv: "f633224303864f52c52025a1a1bf6faf"
server_key_version: "encryptionkey@0"
I20260810 14:38:42.949443 3319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:42.978780 3319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:42.983791 3319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:42.984112 3319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:42.985107 3319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:42.986668 3319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:42.986778 3319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:42.986918 3319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:42.987016 3319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:43.024293 3319 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:35785
I20260810 14:38:43.024341 3505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:35785 every 8 connection(s)
I20260810 14:38:43.025617 3319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:38:43.028592 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 3319
I20260810 14:38:43.028782 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:38:43.029392 21725 external_mini_cluster.cc:1428] Setting key 0398076547d6befc365370974db31196
Aug 10 14:38:43 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372722, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:43.049562 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:43.029440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:39817 (local address 127.21.55.126:33539)
0810 14:38:43.030029 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:43.030036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:43.031227 (+ 1191us) server_negotiation.cc:407] Connection header received
0810 14:38:43.033157 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:43.033162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:43.033247 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:43.033375 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:43.034947 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.035710 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.037255 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.037713 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.037847 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:43.041768 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:43.041774 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:43.041777 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:43.041810 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:43.044862 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:43.046022 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:43.046026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:43.046028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:43.046159 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:43.046875 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:43.046882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:43.046885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:43.047353 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:43.047521 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:43.049036 (+ 1515us) server_negotiation.cc:299] Negotiation successful
0810 14:38:43.049229 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":290,"threads_started":1}
I20260810 14:38:43.051481 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:43.030035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:39817)
0810 14:38:43.031070 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:43.031120 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:43.032880 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:43.033617 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:43.033625 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:43.034154 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:43.034778 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:43.034796 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.035878 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.035882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:43.037105 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:43.037117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.037950 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.037958 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:43.038182 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:43.038943 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:43.038964 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:43.041490 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:43.045039 (+ 3549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:43.045052 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:43.045072 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:43.045851 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:43.046285 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:43.046293 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:43.046299 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:43.046610 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:43.047565 (+ 955us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:43.047711 (+ 146us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:43.048530 (+ 819us) client_negotiation.cc:769] Sending connection context
0810 14:38:43.048870 (+ 340us) client_negotiation.cc:240] Negotiation successful
0810 14:38:43.049533 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":514,"threads_started":1}
I20260810 14:38:43.053720 3507 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33539
I20260810 14:38:43.054189 3507 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:43.055061 3507 heartbeater.cc:511] Master 127.21.55.126:33539 requested a full tablet report, sending...
I20260810 14:38:43.056834 2994 ts_manager.cc:194] Registered new tserver with Master: 16967801d4b149b29f2ee7d1fd58e7e5 (127.21.55.66:35785)
I20260810 14:38:43.058126 2994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:39817
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:38:43.093644 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33539
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:43.452620 3516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:43.452965 3516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:43.453171 3516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:43.460583 3516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:43.460700 3516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:43.460813 3516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:38:43.471450 3516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:43.472743 3516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:43.474012 3516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:38:43.477240 3309 heartbeater.cc:503] Master 127.21.55.126:33539 was elected leader, sending a full tablet report...
W20260810 14:38:43.488720 3525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:43.488751 3526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:43.488834 3530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:43.490027 3516 server_base.cc:1034] running on GCE node
I20260810 14:38:43.490763 3516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:43.491721 3516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:43.492939 3516 hybrid_clock.cc:648] HybridClock initialized: now 1786372723492897 us; error 37 us; skew 500 ppm
I20260810 14:38:43.493268 3516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:43 dist-test-slave-h6n0 krb5kdc[2922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372723, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:43.496613 3516 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:38:43.498236 3516 webserver.cc:460] Webserver started at http://127.21.55.67:42363/ using document root <none> and password file <none>
I20260810 14:38:43.498965 3516 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:43.499076 3516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:43.499320 3516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:38:43.502061 3516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "3786d3166b8b4296b73c9bf474142ddd"
format_stamp: "Formatted at 2026-08-10 14:38:43 on dist-test-slave-h6n0"
server_key: "a531d3a43dd3967a42ab063aa3acd644"
server_key_iv: "a3c4791875e10e5aef2c7af54d876138"
server_key_version: "encryptionkey@0"
I20260810 14:38:43.502866 3516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "3786d3166b8b4296b73c9bf474142ddd"
format_stamp: "Formatted at 2026-08-10 14:38:43 on dist-test-slave-h6n0"
server_key: "a531d3a43dd3967a42ab063aa3acd644"
server_key_iv: "a3c4791875e10e5aef2c7af54d876138"
server_key_version: "encryptionkey@0"
I20260810 14:38:43.507779 3516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 14:38:43.511189 3537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:43.512646 3516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:38:43.512791 3516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "3786d3166b8b4296b73c9bf474142ddd"
format_stamp: "Formatted at 2026-08-10 14:38:43 on dist-test-slave-h6n0"
server_key: "a531d3a43dd3967a42ab063aa3acd644"
server_key_iv: "a3c4791875e10e5aef2c7af54d876138"
server_key_version: "encryptionkey@0"
I20260810 14:38:43.512966 3516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:43.541141 3516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:43.545732 3516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:43.545985 3516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:43.546932 3516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:43.548265 3516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:43.548342 3516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:43.548426 3516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:43.548481 3516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:43.589161 3516 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:33289
I20260810 14:38:43.589479 3719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:33289 every 8 connection(s)
I20260810 14:38:43.590727 3516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:38:43.601181 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 3516
I20260810 14:38:43.601395 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:38:43.601891 21725 external_mini_cluster.cc:1428] Setting key 8f1bf98e17f9bc5068812c108986fc6e
Aug 10 14:38:43 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372723, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:38:43.615036 3726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:43.594791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:57823 (local address 127.21.55.126:33539)
0810 14:38:43.595514 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:43.595520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:43.596813 (+ 1293us) server_negotiation.cc:407] Connection header received
0810 14:38:43.598855 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:43.598861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:43.598990 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:43.599116 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:43.600645 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.601516 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.603478 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.603876 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.604182 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:43.608379 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:43.608385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:43.608390 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:43.608431 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:43.611175 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:43.611918 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:43.611923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:43.611925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:43.612059 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:43.612512 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:43.612524 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:43.612526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:43.612781 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:43.612971 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:43.614286 (+ 1315us) server_negotiation.cc:299] Negotiation successful
0810 14:38:43.614568 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":388,"threads_started":1}
I20260810 14:38:43.616305 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:43.595507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:57823)
0810 14:38:43.596611 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:43.596654 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:43.598517 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:43.599358 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:43.599366 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:43.599899 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:43.600465 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:43.600493 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.601676 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.601680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:43.603317 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:43.603338 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.604269 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.604281 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:43.604568 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:43.605347 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:43.605372 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:43.608160 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:38:43.611326 (+ 3166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:43.611334 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:43.611350 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:43.611777 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:43.612177 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:38:43.612182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:38:43.612186 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:38:43.612383 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:38:43.612927 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:38:43.612935 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:38:43.613534 (+ 599us) client_negotiation.cc:769] Sending connection context
0810 14:38:43.613975 (+ 441us) client_negotiation.cc:240] Negotiation successful
0810 14:38:43.614777 (+ 802us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":409,"threads_started":1}
I20260810 14:38:43.618383 3720 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33539
I20260810 14:38:43.618733 3720 heartbeater.cc:464] Registering TS with master...
I20260810 14:38:43.619421 3720 heartbeater.cc:511] Master 127.21.55.126:33539 requested a full tablet report, sending...
I20260810 14:38:43.620911 2992 ts_manager.cc:194] Registered new tserver with Master: 3786d3166b8b4296b73c9bf474142ddd (127.21.55.67:33289)
I20260810 14:38:43.621853 2992 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:57823
I20260810 14:38:43.631531 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab.
Entry for principal oryx/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab.
WARNING: no policy specified for oryx/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab.
Entry for principal oryx/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab.
WARNING: no policy specified for oryx/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab.
Entry for principal oryx/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab.
I20260810 14:38:43.802680 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 3103
Aug 10 14:38:43 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:38:43.820291 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:43.804101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.21.55.65:37277)
0810 14:38:43.804897 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:43.804905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:43.804931 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:38:43.804997 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:43.805004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:43.805213 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:43.805345 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:43.806772 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.807811 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.809170 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.809743 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.809847 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:43.814351 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:43.814362 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:43.814376 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:43.814407 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:43.817567 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:43.818277 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:43.818284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:43.818286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:43.818408 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:43.818808 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:43.818811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:43.818814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:43.819142 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:43.819372 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:43.819697 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 14:38:43.819890 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":477,"threads_started":1}
I20260810 14:38:43.821511 3299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:43.908229 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 3103
I20260810 14:38:43.940762 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 3319
Aug 10 14:38:43 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:38:43.961110 3744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:43.942082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58170 (local address 127.21.55.66:35785)
0810 14:38:43.943047 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:43.943056 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:43.943094 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:38:43.943228 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:43.943237 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:43.943512 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:43.943696 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:43.945107 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.946348 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.948070 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:43.948726 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:43.949045 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:43.953887 (+ 4842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:43.953904 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:43.953920 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:43.953985 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:43.957325 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:43.958171 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:43.958183 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:43.958185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:43.958445 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:43.959117 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:43.959121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:43.959124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:43.959643 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:43.959966 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:43.960488 (+ 522us) server_negotiation.cc:299] Negotiation successful
0810 14:38:43.960743 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":424,"threads_started":1}
I20260810 14:38:43.962697 3499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:44.048367 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 3319
I20260810 14:38:44.074311 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 3516
Aug 10 14:38:44 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:38:44.094424 3750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:44.075445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.21.55.67:33289)
0810 14:38:44.075949 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:44.075977 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:44.076008 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:38:44.076089 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:44.076095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:44.076315 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:44.076445 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:44.077981 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:44.079192 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:44.080513 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:44.081517 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:44.081633 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:44.087363 (+ 5730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:44.087385 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:44.087407 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:44.087471 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:44.090795 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:44.091484 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:44.091491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:44.091494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:44.091641 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:44.092511 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:44.092515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:44.092517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:44.092916 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:44.093342 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:44.093822 (+ 480us) server_negotiation.cc:299] Negotiation successful
0810 14:38:44.094042 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":231,"threads_started":1}
I20260810 14:38:44.096634 3715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:44.183845 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 3516
I20260810 14:38:44.210676 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 2942
I20260810 14:38:44.212039 3052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:38:44.320109 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 2942
I20260810 14:38:44.346344 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:33539
--webserver_interface=127.21.55.126
--webserver_port=45317
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:33539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:38:44.692991 3759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:44.693321 3759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:44.693426 3759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:44.700944 3759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:38:44.701071 3759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:44.701124 3759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:44.701182 3759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:38:44.701226 3759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:38:44.711616 3759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:33539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:33539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=45317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:44.712932 3759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:44.714215 3759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:44.728142 3768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:44.728288 3771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:44.728215 3769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:44.729743 3759 server_base.cc:1034] running on GCE node
I20260810 14:38:44.730643 3759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:44.732064 3759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:44.733312 3759 hybrid_clock.cc:648] HybridClock initialized: now 1786372724733251 us; error 61 us; skew 500 ppm
I20260810 14:38:44.733747 3759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:44 dist-test-slave-h6n0 krb5kdc[2922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372724, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:44.737855 3759 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:38:44.739565 3759 webserver.cc:460] Webserver started at http://127.21.55.126:45317/ using document root <none> and password file <none>
I20260810 14:38:44.740293 3759 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:44.740470 3759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:44.746646 3759 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:38:44.750921 3785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:44.752676 3759 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 14:38:44.752897 3759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "5425face473c401bb2421dccabe167df"
format_stamp: "Formatted at 2026-08-10 14:38:41 on dist-test-slave-h6n0"
server_key: "129f4ab3e3df1191136cb870098a0438"
server_key_iv: "39bd6abe5fe3a8f3bab36c8250b10fca"
server_key_version: "encryptionkey@0"
I20260810 14:38:44.753938 3759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:44.782800 3759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:44.788352 3759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:44.788651 3759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:44.811033 3759 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:33539
I20260810 14:38:44.811070 3874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:33539 every 8 connection(s)
I20260810 14:38:44.812649 3759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:38:44.815236 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 3759
I20260810 14:38:44.816300 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:37277
--local_ip_for_outbound_sockets=127.21.55.65
--tserver_master_addrs=127.21.55.126:33539
--webserver_port=39171
--webserver_interface=127.21.55.65
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.55.65
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:38:44.822154 3877 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:38:44.827262 3877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df: Bootstrap starting.
I20260810 14:38:44.846505 3877 log.cc:824] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df: Log is configured to *not* fsync() on all Append() calls
I20260810 14:38:44.856698 3877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:38:44.857583 3877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df: Bootstrap complete.
I20260810 14:38:44.869850 3877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } }
I20260810 14:38:44.871290 3877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5425face473c401bb2421dccabe167df, State: Initialized, Role: FOLLOWER
I20260810 14:38:44.872166 3877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } }
I20260810 14:38:44.872481 3877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:38:44.872618 3877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:38:44.872857 3877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:38:44.877674 3877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } }
I20260810 14:38:44.878311 3877 leader_election.cc:302] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5425face473c401bb2421dccabe167df; no voters:
I20260810 14:38:44.879058 3877 leader_election.cc:288] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:38:44.879207 3882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:38:44.880509 3882 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [term 2 LEADER]: Becoming Leader. State: Replica: 5425face473c401bb2421dccabe167df, State: Running, Role: LEADER
I20260810 14:38:44.881176 3882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } }
I20260810 14:38:44.881426 3877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:38:44.884045 3883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5425face473c401bb2421dccabe167df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } } }
I20260810 14:38:44.884495 3883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:44.884644 3885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5425face473c401bb2421dccabe167df. Latest consensus state: current_term: 2 leader_uuid: "5425face473c401bb2421dccabe167df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5425face473c401bb2421dccabe167df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33539 } } }
I20260810 14:38:44.885057 3885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df [sys.catalog]: This master's current role is: LEADER
I20260810 14:38:44.897071 3890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:38:44.903746 3890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:38:44.905352 3890 catalog_manager.cc:1259] Loaded cluster ID: 435fccfb08c54f1ebab571177bfcf313
I20260810 14:38:44.905530 3890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:38:44.910045 3890 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:38:44.911860 3890 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5425face473c401bb2421dccabe167df: Loaded TSK: 0
I20260810 14:38:44.913516 3890 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:38:45.214184 3879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:45.214610 3879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:45.214785 3879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:45.222221 3879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:45.222384 3879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:45.222625 3879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:38:45.232678 3879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.65.keytab
--principal=oryx/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:37277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=39171
--tserver_master_addrs=127.21.55.126:33539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:45.234040 3879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:45.235512 3879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:45.250514 3909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:45.250488 3915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:45.250739 3912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:45.251626 3879 server_base.cc:1034] running on GCE node
I20260810 14:38:45.252276 3879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:45.253335 3879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:45.254619 3879 hybrid_clock.cc:648] HybridClock initialized: now 1786372725254547 us; error 51 us; skew 500 ppm
I20260810 14:38:45.255071 3879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:45 dist-test-slave-h6n0 krb5kdc[2922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372725, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:45.259552 3879 init.cc:377] Logged in from keytab as oryx/127.21.55.65@KRBTEST.COM (short username oryx)
I20260810 14:38:45.261987 3879 webserver.cc:460] Webserver started at http://127.21.55.65:39171/ using document root <none> and password file <none>
I20260810 14:38:45.263145 3879 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:45.263301 3879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:45.269058 3879 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:38:45.272404 3927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:45.274102 3879 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260810 14:38:45.274250 3879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "9215509df28d4acc8216b9345c99d483"
format_stamp: "Formatted at 2026-08-10 14:38:42 on dist-test-slave-h6n0"
server_key: "0ecb6e423683b5c7e1eddb802125e7df"
server_key_iv: "7851c078ed8953ba1d80bbd805daaddb"
server_key_version: "encryptionkey@0"
I20260810 14:38:45.275295 3879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:45.296823 3879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:45.302124 3879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:45.302364 3879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:45.303575 3879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:45.305020 3879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:45.305126 3879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:45.305222 3879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:45.305280 3879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:45.344448 3879 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:37277
I20260810 14:38:45.344605 4111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:37277 every 8 connection(s)
I20260810 14:38:45.346792 3879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:38:45.347929 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 3879
I20260810 14:38:45.349094 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:35785
--local_ip_for_outbound_sockets=127.21.55.66
--tserver_master_addrs=127.21.55.126:33539
--webserver_port=45373
--webserver_interface=127.21.55.66
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.55.66
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:38:45 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:45 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:38:45.723577 4120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:45.723874 4120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:45.723971 4120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:45.731654 4120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:45.731777 4120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:45.731952 4120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:38:45.742146 4120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.66.keytab
--principal=oryx/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:35785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=45373
--tserver_master_addrs=127.21.55.126:33539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:45.743584 4120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:45.744948 4120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:45.761106 4127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:45.761297 4126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:45.762017 4131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:45.762511 4120 server_base.cc:1034] running on GCE node
I20260810 14:38:45.763700 4120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:45.765347 4120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:45.766654 4120 hybrid_clock.cc:648] HybridClock initialized: now 1786372725766562 us; error 54 us; skew 500 ppm
I20260810 14:38:45.767195 4120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:45 dist-test-slave-h6n0 krb5kdc[2922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372725, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:45.772850 4120 init.cc:377] Logged in from keytab as oryx/127.21.55.66@KRBTEST.COM (short username oryx)
I20260810 14:38:45.775563 4120 webserver.cc:460] Webserver started at http://127.21.55.66:45373/ using document root <none> and password file <none>
I20260810 14:38:45.777202 4120 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:45.777796 4120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:45.788611 4120 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.003s
I20260810 14:38:45.793743 4142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:45.796176 4120 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260810 14:38:45.796535 4120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "16967801d4b149b29f2ee7d1fd58e7e5"
format_stamp: "Formatted at 2026-08-10 14:38:42 on dist-test-slave-h6n0"
server_key: "29b22d4f6dfc94d61c795abd67993bbc"
server_key_iv: "f633224303864f52c52025a1a1bf6faf"
server_key_version: "encryptionkey@0"
I20260810 14:38:45.798801 4120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:45.845551 4120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:45.851058 4120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:45.851356 4120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:45.852515 4120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:45.854110 4120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:45.854209 4120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:45.854362 4120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:45.854439 4120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:45.907014 4120 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:35785
I20260810 14:38:45.907071 4321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:35785 every 8 connection(s)
I20260810 14:38:45.908564 4120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:38:45.915611 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 4120
I20260810 14:38:45.916288 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:33289
--local_ip_for_outbound_sockets=127.21.55.67
--tserver_master_addrs=127.21.55.126:33539
--webserver_port=42363
--webserver_interface=127.21.55.67
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.55.67
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:38:45 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:45 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:38:46.293802 4329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:38:46.294109 4329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:38:46.294258 4329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:38:46.301719 4329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:38:46.301872 4329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:38:46.302054 4329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:38:46.312023 4329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/krb5kdc/oryx_127.21.55.67.keytab
--principal=oryx/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:33289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=42363
--tserver_master_addrs=127.21.55.126:33539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:38:46.313298 4329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:38:46.314596 4329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:38:46.331769 4339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:46.331926 4338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:38:46.340147 4341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:38:46.340490 4329 server_base.cc:1034] running on GCE node
I20260810 14:38:46.341358 4329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:38:46.342453 4329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:38:46.344354 4329 hybrid_clock.cc:648] HybridClock initialized: now 1786372726344240 us; error 121 us; skew 500 ppm
I20260810 14:38:46.345009 4329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:38:46 dist-test-slave-h6n0 krb5kdc[2922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372726, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:38:46.354844 4329 init.cc:377] Logged in from keytab as oryx/127.21.55.67@KRBTEST.COM (short username oryx)
I20260810 14:38:46.367122 4329 webserver.cc:460] Webserver started at http://127.21.55.67:42363/ using document root <none> and password file <none>
I20260810 14:38:46.368515 4329 fs_manager.cc:359] Metadata directory not provided
I20260810 14:38:46.368741 4329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:38:46.379729 4329 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 14:38:46.384888 4350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:46.387331 4329 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:38:46.387611 4329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "3786d3166b8b4296b73c9bf474142ddd"
format_stamp: "Formatted at 2026-08-10 14:38:43 on dist-test-slave-h6n0"
server_key: "a531d3a43dd3967a42ab063aa3acd644"
server_key_iv: "a3c4791875e10e5aef2c7af54d876138"
server_key_version: "encryptionkey@0"
I20260810 14:38:46.389979 4329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:38:46.431474 4329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:38:46.439383 4329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:38:46.439802 4329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:38:46.441550 4329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:38:46.444120 4329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:38:46.444285 4329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:46.444442 4329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:38:46.444555 4329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:38:46.522691 4329 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:33289
I20260810 14:38:46.522982 4541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:33289 every 8 connection(s)
I20260810 14:38:46.525223 4329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:38:46.531314 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 4329
Aug 10 14:38:46 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:46 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:46.572328 4549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:46.543983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35124 (local address 127.21.55.126:33539)
0810 14:38:46.544535 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:46.544541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:46.544563 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:38:46.544629 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:46.544635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:46.544715 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:46.547190 (+ 2475us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:46.547410 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.548723 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.550860 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.551525 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.555016 (+ 3491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:46.556852 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:38:46.556866 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:38:46.556886 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:38:46.556934 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:38:46.560824 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:46.566711 (+ 5887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:46.566724 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:46.566737 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:46.567004 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:38:46.567534 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:38:46.567540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:38:46.567544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:38:46.568332 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:38:46.570133 (+ 1801us) server_negotiation.cc:1035] Waiting for connection context
0810 14:38:46.570485 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 14:38:46.571053 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":284,"threads_started":1}
W20260810 14:38:46.938551 4118 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.576s user 0.002s sys 0.014s
W20260810 14:38:46.940811 4117 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:45.351083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:36965 (local address 127.21.55.126:33539)
0810 14:38:45.351838 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:45.351867 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:45.352359 (+ 492us) server_negotiation.cc:407] Connection header received
0810 14:38:45.354578 (+ 2219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:45.354597 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:45.354937 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:45.355272 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:45.356810 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:45.358290 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:45.359969 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:45.360576 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:45.360778 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:46.940327 (+1579549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:36965: BlockingRecv error: recv got EOF from 127.21.55.65:36965 (error 108)
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":307,"threads_started":1}
I20260810 14:38:46.941524 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:45.351383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:36965)
0810 14:38:45.352191 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:45.352227 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:45.354008 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:45.355434 (+ 1426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:45.355444 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:45.356072 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:45.356594 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:45.356610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:45.358463 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:45.358467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:45.359811 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:45.359823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:45.360826 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:45.360834 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:45.361099 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:45.361925 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:45.361953 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:46.939879 (+1577926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":366,"threads_started":1}
W20260810 14:38:46.943153 4113 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:33539 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.55.126:33539: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Aug 10 14:38:46 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:46 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:38:46.961504 4117 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:46.944247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:50759 (local address 127.21.55.126:33539)
0810 14:38:46.944499 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:46.944509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:46.944651 (+ 142us) server_negotiation.cc:407] Connection header received
0810 14:38:46.944873 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:46.944881 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:46.944992 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:46.945145 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:46.946422 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.948222 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.951077 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.952031 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.952206 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:46.961077 (+ 8871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:50759: BlockingRecv error: recv got EOF from 127.21.55.65:50759 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:38:46.963302 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:46.944146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:50759)
0810 14:38:46.944521 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:46.944544 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:46.944718 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:46.945208 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:46.945216 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:46.945913 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:46.946220 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:46.946237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.948500 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.948509 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:46.950871 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:46.950888 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.952309 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.952319 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:46.952619 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:46.953647 (+ 1028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:46.953685 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:46.962545 (+ 8860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186}
Aug 10 14:38:46 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:46 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:46.983165 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:46.964665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:34807)
0810 14:38:46.964910 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:46.964940 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:46.965084 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:46.965890 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:46.965897 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:46.966496 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:46.966824 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:46.966841 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.969187 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.969195 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:46.972085 (+ 2890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:46.972109 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.973682 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.973691 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:46.974389 (+ 698us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:46.975618 (+ 1229us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:46.975656 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:46.982624 (+ 6968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76}
W20260810 14:38:46.983458 4117 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:46.965080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:34807 (local address 127.21.55.126:33539)
0810 14:38:46.965290 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:46.965299 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:46.965322 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:38:46.965393 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:46.965401 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:46.965659 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:46.965846 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:46.967041 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.968960 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.972301 (+ 3341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.973451 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.973650 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:46.983052 (+ 9402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:34807: BlockingRecv error: recv got EOF from 127.21.55.65:34807 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 14:38:46.983953 4113 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:38:47.293951 4327 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.364s user 0.008s sys 0.013s
I20260810 14:38:47.295696 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:45.912542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:45245)
0810 14:38:45.913684 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:45.913764 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:45.917563 (+ 3799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:45.918736 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:45.918755 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:45.919927 (+ 1172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:45.921125 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:45.921162 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:45.922911 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:45.922921 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:45.925959 (+ 3038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:45.925982 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:45.927582 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:45.927591 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:45.927933 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:45.929565 (+ 1632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:45.929621 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:47.294828 (+1365207us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":416,"threads_started":1}
W20260810 14:38:47.295965 4326 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:45.912198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:45245 (local address 127.21.55.126:33539)
0810 14:38:45.912796 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:45.912804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:45.913936 (+ 1132us) server_negotiation.cc:407] Connection header received
0810 14:38:45.918033 (+ 4097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:45.918042 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:45.918144 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:45.918275 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:45.921411 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:45.922682 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:45.926264 (+ 3582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:45.927287 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:45.927480 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.295338 (+1367858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:45245: BlockingRecv error: recv got EOF from 127.21.55.66:45245 (error 108)
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":309,"threads_started":1}
W20260810 14:38:47.297008 4323 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:33539 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.55.126:33539: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Aug 10 14:38:47 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:47 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:47.311220 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:47.297991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:39475)
0810 14:38:47.298749 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:47.298770 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:47.298908 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:47.299453 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:47.299456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:47.299822 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:47.299986 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:47.299993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.302147 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.302151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:47.303399 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:47.303412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.304867 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.304872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:47.305002 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.305861 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:47.305878 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:47.310908 (+ 5030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":607}
W20260810 14:38:47.311368 4326 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:47.297919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:39475 (local address 127.21.55.126:33539)
0810 14:38:47.298100 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:47.298108 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:47.298881 (+ 773us) server_negotiation.cc:407] Connection header received
0810 14:38:47.299126 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:47.299136 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:47.299264 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:47.299413 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:47.300150 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.301940 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.303572 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.304637 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.304857 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.311108 (+ 6251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:39475: BlockingRecv error: recv got EOF from 127.21.55.66:39475 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 14:38:47 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:47 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:47.322589 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:47.312265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:39511)
0810 14:38:47.312595 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:47.312613 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:47.312716 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:47.313051 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:47.313056 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:47.313396 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:47.313546 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:47.313554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.314718 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.314724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:47.315670 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:47.315681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.316418 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.316424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:47.316621 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.317461 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:47.317480 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:47.322206 (+ 4726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225}
W20260810 14:38:47.322889 4326 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:47.312423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:39511 (local address 127.21.55.126:33539)
0810 14:38:47.312570 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:47.312576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:47.312660 (+ 84us) server_negotiation.cc:407] Connection header received
0810 14:38:47.312842 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:47.312847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:47.312919 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:47.313029 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:47.313684 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.314522 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.315814 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.316259 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.316401 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.322566 (+ 6165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:39511: BlockingRecv error: recv got EOF from 127.21.55.66:39511 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260810 14:38:47.323140 4323 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:38:47.709969 4548 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.155s user 0.007s sys 0.011s
W20260810 14:38:47.711292 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:46.532282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:58989 (local address 127.21.55.126:33539)
0810 14:38:46.544056 (+ 11774us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:46.544063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:46.544090 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:38:46.546250 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:46.546258 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:46.546353 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:46.546447 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:46.549001 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.550244 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.552905 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.553617 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.553736 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.711004 (+1157268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:58989: BlockingRecv error: recv got EOF from 127.21.55.67:58989 (error 108)
Metrics: {"server-negotiator.queue_time_us":11602,"thread_start_us":11415,"threads_started":1}
I20260810 14:38:47.711885 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:46.534990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:58989)
0810 14:38:46.543165 (+ 8175us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:46.543215 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:46.545878 (+ 2663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:46.547122 (+ 1244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:46.547136 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:46.547975 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:46.548799 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:46.548829 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.550492 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.550497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:46.552706 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:46.552727 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:46.554160 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:46.554169 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:46.554551 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:46.555380 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:46.555418 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:47.710793 (+1155375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":7846,"thread_start_us":7554,"threads_started":1}
W20260810 14:38:47.712821 4542 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:33539 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.55.126:33539: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Aug 10 14:38:47 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:47 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:47.728308 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:47.713516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:46729)
0810 14:38:47.713759 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:47.713781 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:47.713902 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:47.714308 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:47.714313 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:47.714840 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:47.715017 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:47.715025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.716208 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.716212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:47.717186 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:47.717198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.717998 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.718003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:47.718135 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.723522 (+ 5387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:47.723541 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:47.727962 (+ 4421us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":81}
W20260810 14:38:47.728948 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:47.713677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:46729 (local address 127.21.55.126:33539)
0810 14:38:47.713846 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:47.713850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:47.713866 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:38:47.714047 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:47.714055 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:47.714136 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:47.714222 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:47.715178 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.715941 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.717335 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.717772 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.717872 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.728456 (+ 10584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:46729: BlockingRecv error: recv got EOF from 127.21.55.67:46729 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 14:38:47 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:47 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:47.738989 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:47.729019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:34097)
0810 14:38:47.729259 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:47.729277 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:47.729374 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:47.730062 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:47.730066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:47.730359 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:47.730511 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:47.730519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.732052 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.732055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:47.733061 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:47.733072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.733968 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.733973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:47.734109 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.734801 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:47.734817 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:47.738664 (+ 3847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134}
W20260810 14:38:47.739221 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:47.729408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:34097 (local address 127.21.55.126:33539)
0810 14:38:47.729781 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:47.729786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:47.729802 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:38:47.729845 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:47.729849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:47.729923 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:47.730011 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:47.730707 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.731856 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.733237 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.733728 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.733939 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.738933 (+ 4994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:34097: BlockingRecv error: recv got EOF from 127.21.55.67:34097 (error 108)
Metrics: {"server-negotiator.queue_time_us":237}
W20260810 14:38:47.739336 4542 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 14:38:47 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:47 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:47.996945 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:47.984866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:57065)
0810 14:38:47.985521 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:47.985545 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:47.985701 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:47.986107 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:47.986112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:47.986730 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:47.986897 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:47.986907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.988651 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.988657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:47.989684 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:47.989697 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.990896 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.990903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:47.991115 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.991864 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:47.991885 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:47.996586 (+ 4701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":343,"threads_started":1}
W20260810 14:38:47.997028 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:47.984681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:57065 (local address 127.21.55.126:33539)
0810 14:38:47.984885 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:47.984894 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:47.985621 (+ 727us) server_negotiation.cc:407] Connection header received
0810 14:38:47.985859 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:47.985865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:47.985951 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:47.986068 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:47.987062 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.988461 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.989856 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:47.990686 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:47.990842 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:47.996778 (+ 5936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:57065: BlockingRecv error: recv got EOF from 127.21.55.65:57065 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 14:38:48 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:48 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:48.338140 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:48.324145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:37091)
0810 14:38:48.324957 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:48.324979 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:48.325131 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:48.325700 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:48.325704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:48.326199 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:48.326484 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:48.326501 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:48.328209 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:48.328213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:48.329678 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:48.329700 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:48.331018 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:48.331032 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:48.331280 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:48.332019 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:48.332044 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:48.337565 (+ 5521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":361,"threads_started":1}
W20260810 14:38:48.338446 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:48.324221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:37091 (local address 127.21.55.126:33539)
0810 14:38:48.324456 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:48.324463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:48.325116 (+ 653us) server_negotiation.cc:407] Connection header received
0810 14:38:48.325433 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:48.325438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:48.325513 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:48.325609 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:48.326830 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:48.328010 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:48.330062 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:48.330521 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:48.330715 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:48.337801 (+ 7086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:37091: BlockingRecv error: recv got EOF from 127.21.55.66:37091 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 14:38:48 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:48 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:48.753575 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:48.740250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:33159)
0810 14:38:48.741186 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:48.741206 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:48.741343 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:48.741859 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:48.741863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:48.742358 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:48.742518 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:48.742590 (+ 72us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:48.743806 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:48.743812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:48.745045 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:48.745061 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:48.746010 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:48.746017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:48.746221 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:48.747459 (+ 1238us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:48.747486 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:48.753069 (+ 5583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":793,"thread_start_us":625,"threads_started":1}
W20260810 14:38:48.753912 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:48.740373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:33159 (local address 127.21.55.126:33539)
0810 14:38:48.740632 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:48.740638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:48.741331 (+ 693us) server_negotiation.cc:407] Connection header received
0810 14:38:48.741546 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:48.741550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:48.741638 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:48.741768 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:48.742808 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:48.743579 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:48.745212 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:48.745720 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:48.745843 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:48.753609 (+ 7766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:33159: BlockingRecv error: recv got EOF from 127.21.55.67:33159 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 10 14:38:48 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:49 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:49.009610 4558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:48.998188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:41265)
0810 14:38:48.999180 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:48.999203 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:48.999407 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:48.999833 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:48.999839 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:49.000329 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:49.000523 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:49.000532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.001586 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.001590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:49.002589 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:49.002601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.003333 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.003339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:49.003544 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:49.004372 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:49.004401 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:49.009222 (+ 4821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":839,"thread_start_us":656,"threads_started":1}
W20260810 14:38:49.009829 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:48.998434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:41265 (local address 127.21.55.126:33539)
0810 14:38:48.998762 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:48.998773 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:48.999361 (+ 588us) server_negotiation.cc:407] Connection header received
0810 14:38:48.999516 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:48.999527 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:48.999601 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:48.999694 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:49.000696 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.001431 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.002752 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.003169 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.003306 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:49.009529 (+ 6223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:41265: BlockingRecv error: recv got EOF from 127.21.55.65:41265 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Aug 10 14:38:49 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:49 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:49.351998 4559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:49.339688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:60645)
0810 14:38:49.340511 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:49.340533 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:49.340693 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:49.341261 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:49.341265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:49.341726 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:49.341922 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:49.341930 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.343361 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.343366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:49.344443 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:49.344455 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.345550 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.345558 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:49.345834 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:49.346866 (+ 1032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:49.346896 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:49.351675 (+ 4779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":368,"threads_started":1}
W20260810 14:38:49.352135 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:49.340100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:60645 (local address 127.21.55.126:33539)
0810 14:38:49.340306 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:49.340311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:49.340598 (+ 287us) server_negotiation.cc:407] Connection header received
0810 14:38:49.340914 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:49.340923 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:49.341000 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:49.341169 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:49.342132 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.343191 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.344750 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.345273 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.345382 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:49.351917 (+ 6535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:60645: BlockingRecv error: recv got EOF from 127.21.55.66:60645 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 10 14:38:49 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:49 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:49.764047 4563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:49.754731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:54835)
0810 14:38:49.755298 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:49.755316 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:49.755434 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:49.755742 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:49.755745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:49.756154 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:49.756334 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:49.756342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.757393 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.757396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:49.758410 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:49.758430 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.759189 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.759194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:49.759316 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:49.759977 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:49.759996 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:49.763655 (+ 3659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":345,"threads_started":1}
W20260810 14:38:49.764153 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:49.754955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:54835 (local address 127.21.55.126:33539)
0810 14:38:49.755250 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:49.755256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:49.755392 (+ 136us) server_negotiation.cc:407] Connection header received
0810 14:38:49.755542 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:49.755545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:49.755607 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:49.755687 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:49.756461 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.757271 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.758594 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:49.759046 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:49.759139 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:49.763851 (+ 4712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:54835: BlockingRecv error: recv got EOF from 127.21.55.67:54835 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
Aug 10 14:38:50 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:50 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:50.022293 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:50.010835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:48619)
0810 14:38:50.011366 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:50.011383 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:50.011568 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:50.011925 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:50.011929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:50.012333 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:50.012493 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:50.012501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.013633 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.013638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:50.014597 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:50.014620 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.015504 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.015666 (+ 162us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:50.015914 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:50.016668 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:50.016702 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:50.021860 (+ 5158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":291,"threads_started":1}
W20260810 14:38:50.022353 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:50.011054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:48619 (local address 127.21.55.126:33539)
0810 14:38:50.011260 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:50.011266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:50.011479 (+ 213us) server_negotiation.cc:407] Connection header received
0810 14:38:50.011703 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:50.011708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:50.011781 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:50.011891 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:50.012671 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.013472 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.014749 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.015306 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.015456 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:50.022121 (+ 6665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:48619: BlockingRecv error: recv got EOF from 127.21.55.65:48619 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 10 14:38:50 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:50 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:50.364703 4568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:50.353174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:49129)
0810 14:38:50.353722 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:50.353741 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:50.353870 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:50.354231 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:50.354235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:50.354704 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:50.354996 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:50.355011 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.356136 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.356140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:50.357227 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:50.357240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.358275 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.358284 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:50.358434 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:50.359330 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:50.359357 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:50.364393 (+ 5036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":295,"threads_started":1}
W20260810 14:38:50.364926 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:50.353288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:49129 (local address 127.21.55.126:33539)
0810 14:38:50.353565 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:50.353572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:50.353810 (+ 238us) server_negotiation.cc:407] Connection header received
0810 14:38:50.353999 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:50.354002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:50.354065 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:50.354148 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:50.355166 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.355947 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.357406 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.358041 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.358172 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:50.364650 (+ 6478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:49129: BlockingRecv error: recv got EOF from 127.21.55.66:49129 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Aug 10 14:38:50 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:50 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:50.775734 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:50.765156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:45855)
0810 14:38:50.765771 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:50.765789 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:50.765991 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:50.766601 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:50.766610 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:50.767129 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:50.767310 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:50.767320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.768383 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.768387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:50.769668 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:50.769683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.770691 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.770698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:50.770876 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:50.771620 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:50.771647 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:50.775380 (+ 3733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":343,"threads_started":1}
W20260810 14:38:50.775825 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:50.765471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:45855 (local address 127.21.55.126:33539)
0810 14:38:50.765693 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:50.765698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:50.765949 (+ 251us) server_negotiation.cc:407] Connection header received
0810 14:38:50.766185 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:50.766190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:50.766273 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:50.766362 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:50.767466 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.768235 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.769852 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:50.770496 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:50.770660 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:50.775607 (+ 4947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:45855: BlockingRecv error: recv got EOF from 127.21.55.67:45855 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 10 14:38:51 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:51 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:51.034191 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:51.023596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:36327)
0810 14:38:51.024325 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:51.024345 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:51.024500 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:51.024973 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:51.024978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:51.025450 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:51.025634 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:51.025644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.026783 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.026794 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:51.027817 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:51.027830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.028539 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.028543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:51.028670 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:51.029368 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:51.029386 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:51.033793 (+ 4407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":387,"threads_started":1}
W20260810 14:38:51.034475 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:51.023743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:36327 (local address 127.21.55.126:33539)
0810 14:38:51.023939 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:51.023944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:51.024467 (+ 523us) server_negotiation.cc:407] Connection header received
0810 14:38:51.024623 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:51.024627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:51.024697 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:51.024817 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:51.025802 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.026633 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.027971 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.028402 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.028503 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:51.034032 (+ 5529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:36327: BlockingRecv error: recv got EOF from 127.21.55.65:36327 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 10 14:38:51 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:51 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:51.376453 4571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:51.366213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:57279)
0810 14:38:51.366803 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:51.366821 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:51.366928 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:51.367471 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:51.367474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:51.367898 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:51.368078 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:51.368086 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.369189 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.369193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:51.370709 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:51.370725 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.371728 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.371735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:51.371938 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:51.372555 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:51.372573 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:51.376081 (+ 3508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":332,"threads_started":1}
W20260810 14:38:51.376543 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:51.366407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:57279 (local address 127.21.55.126:33539)
0810 14:38:51.366907 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:51.366913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:51.366942 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:38:51.367022 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:51.367026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:51.367248 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:51.367385 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:51.368275 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.369040 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.370856 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.371525 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.371686 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:51.376301 (+ 4615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:57279: BlockingRecv error: recv got EOF from 127.21.55.66:57279 (error 108)
Metrics: {"server-negotiator.queue_time_us":329}
Aug 10 14:38:51 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:51 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:51.788528 4573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:51.776877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:43537)
0810 14:38:51.777716 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:51.777741 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:51.777918 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:51.778680 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:51.778685 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:51.779160 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:51.779339 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:51.779348 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.780594 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.780599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:51.781895 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:51.781909 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.782761 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.782766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:51.783294 (+ 528us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:51.784570 (+ 1276us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:51.784589 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:51.788243 (+ 3654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":372,"threads_started":1}
W20260810 14:38:51.788606 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:51.776972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:43537 (local address 127.21.55.126:33539)
0810 14:38:51.777244 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:51.777254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:51.777909 (+ 655us) server_negotiation.cc:407] Connection header received
0810 14:38:51.778218 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:51.778225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:51.778327 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:51.778499 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:51.779508 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.780442 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.782048 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:51.782616 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:51.782712 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:51.788410 (+ 5698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:43537: BlockingRecv error: recv got EOF from 127.21.55.67:43537 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 10 14:38:52 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:52 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:52.046989 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:52.035619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:48723)
0810 14:38:52.036181 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:52.036199 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:52.036324 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:52.036621 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:52.036624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:52.037093 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:52.037258 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:52.037266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.038271 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.038275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:52.039647 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:52.039661 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.040399 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.040405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:52.040570 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:52.041328 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:52.041348 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:52.046615 (+ 5267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":311,"threads_started":1}
W20260810 14:38:52.047159 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:52.035697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:48723 (local address 127.21.55.126:33539)
0810 14:38:52.035883 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:52.035887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:52.036286 (+ 399us) server_negotiation.cc:407] Connection header received
0810 14:38:52.036414 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:52.036417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:52.036483 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:52.036578 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:52.037399 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.038140 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.039807 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.040228 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.040332 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:52.046829 (+ 6497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:48723: BlockingRecv error: recv got EOF from 127.21.55.65:48723 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 14:38:52 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:52 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:52.386898 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:52.377580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:47839)
0810 14:38:52.378175 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:52.378195 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:52.378321 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:52.378641 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:52.378645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:52.379139 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:52.379322 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:52.379335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.380463 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.380471 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:52.381546 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:52.381558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.382344 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.382350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:52.382488 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:52.383103 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:52.383120 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:52.386639 (+ 3519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":291,"threads_started":1}
W20260810 14:38:52.387102 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:52.377849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:47839 (local address 127.21.55.126:33539)
0810 14:38:52.378058 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:52.378065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:52.378283 (+ 218us) server_negotiation.cc:407] Connection header received
0810 14:38:52.378417 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:52.378421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:52.378483 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:52.378602 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:52.379535 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.380293 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.381684 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.382160 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.382259 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:52.386830 (+ 4571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:47839: BlockingRecv error: recv got EOF from 127.21.55.66:47839 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 10 14:38:52 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:52 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:52.805404 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:52.789511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:44537)
0810 14:38:52.790074 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:52.790090 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:52.790210 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:52.790582 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:52.790592 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:52.791046 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:52.791195 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:52.791204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.792650 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.792654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:52.793632 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:52.793643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.794570 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.794577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:52.794803 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:52.800408 (+ 5605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:52.800431 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:52.805152 (+ 4721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":315,"threads_started":1}
W20260810 14:38:52.805670 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:52.789628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:44537 (local address 127.21.55.126:33539)
0810 14:38:52.789801 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:52.789806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:52.790204 (+ 398us) server_negotiation.cc:407] Connection header received
0810 14:38:52.790326 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:52.790329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:52.790394 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:52.790482 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:52.791469 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.792428 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.793821 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:52.794280 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:52.794393 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:52.805400 (+ 11007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:44537: BlockingRecv error: recv got EOF from 127.21.55.67:44537 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 14:38:53 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:53 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:53.059448 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:53.048512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:39559)
0810 14:38:53.049096 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:53.049114 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:53.049235 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:53.049621 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:53.049624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:53.050070 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:53.050242 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:53.050251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.051560 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.051564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:53.052499 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:53.052511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.053253 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.053258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:53.053395 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:53.054317 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:53.054341 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:53.059203 (+ 4862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":345,"threads_started":1}
W20260810 14:38:53.060032 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:53.048777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:39559 (local address 127.21.55.126:33539)
0810 14:38:53.048979 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:53.048984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:53.049268 (+ 284us) server_negotiation.cc:407] Connection header received
0810 14:38:53.049358 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:53.049363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:53.049453 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:53.049576 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:53.050411 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.051406 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.052646 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.053077 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.053179 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:53.059704 (+ 6525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:39559: BlockingRecv error: recv got EOF from 127.21.55.65:39559 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 14:38:53 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:53 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:53.399710 4584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:53.388085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:44909)
0810 14:38:53.388640 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:53.388656 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:53.388766 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:53.389100 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:53.389104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:53.389546 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:53.389721 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:53.389729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.391198 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.391202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:53.392293 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:53.392306 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.393213 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.393220 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:53.393413 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:53.394249 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:53.394267 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:53.399398 (+ 5131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":315,"threads_started":1}
W20260810 14:38:53.399840 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:53.388217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:44909 (local address 127.21.55.126:33539)
0810 14:38:53.388653 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:53.388657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:53.388730 (+ 73us) server_negotiation.cc:407] Connection header received
0810 14:38:53.388863 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:53.388867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:53.388933 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:53.389034 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:53.390134 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.390904 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.392467 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.393033 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.393180 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:53.399627 (+ 6447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:44909: BlockingRecv error: recv got EOF from 127.21.55.66:44909 (error 108)
Metrics: {"server-negotiator.queue_time_us":316}
Aug 10 14:38:53 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:53 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:53.816418 4585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:53.806373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:44593)
0810 14:38:53.806995 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:53.807012 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:53.807174 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:53.807585 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:53.807589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:53.808072 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:53.808236 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:53.808245 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.809298 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.809305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:53.810227 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:53.810238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.811022 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.811029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:53.811208 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:53.811885 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:53.811908 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:53.816072 (+ 4164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"spinlock_wait_cycles":30080,"thread_start_us":379,"threads_started":1}
W20260810 14:38:53.816613 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:53.806550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:44593 (local address 127.21.55.126:33539)
0810 14:38:53.806766 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:53.806774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:53.807103 (+ 329us) server_negotiation.cc:407] Connection header received
0810 14:38:53.807340 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:53.807345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:53.807415 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:53.807522 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:53.808390 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.809154 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.810375 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:53.810858 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:53.810998 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:53.816321 (+ 5323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:44593: BlockingRecv error: recv got EOF from 127.21.55.67:44593 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 14:38:54 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:54 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:38:54.071568 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:54.060673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:43317 (local address 127.21.55.126:33539)
0810 14:38:54.060958 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:54.060965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:54.061362 (+ 397us) server_negotiation.cc:407] Connection header received
0810 14:38:54.061474 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:54.061479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:54.061580 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:54.061707 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:54.062687 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.063421 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.064784 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.065289 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.065427 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:54.071364 (+ 5937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:43317: BlockingRecv error: recv got EOF from 127.21.55.65:43317 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:38:54.071688 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:54.060528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:43317)
0810 14:38:54.061168 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:54.061185 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:54.061308 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:54.061747 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:54.061750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:54.062257 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:54.062467 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:54.062476 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.063586 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.063590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:54.064618 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:54.064631 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.065461 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.065465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:54.065576 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:54.066221 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:54.066246 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:54.071200 (+ 4954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":291,"threads_started":1}
Aug 10 14:38:54 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:54 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:54.412981 4587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:54.401250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:42043)
0810 14:38:54.402101 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:54.402123 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:54.402280 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:54.402766 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:54.402770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:54.403384 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:54.403639 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:54.403651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.405162 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.405170 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:54.406753 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:54.406771 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.407808 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.407813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:54.407975 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:54.408886 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:54.408910 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:54.412706 (+ 3796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":424,"threads_started":1}
W20260810 14:38:54.413260 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:54.401442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:42043 (local address 127.21.55.126:33539)
0810 14:38:54.401765 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:54.401771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:54.402249 (+ 478us) server_negotiation.cc:407] Connection header received
0810 14:38:54.402439 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:54.402444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:54.402550 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:54.402659 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:54.403872 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.404963 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.406947 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.407601 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.407733 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:54.412965 (+ 5232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:42043: BlockingRecv error: recv got EOF from 127.21.55.66:42043 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 10 14:38:54 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:54 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:54.826493 4588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:54.817483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:54335)
0810 14:38:54.818245 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:54.818261 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:54.818386 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:54.818791 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:54.818794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:54.819211 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:54.819391 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:54.819399 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.820417 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.820420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:54.821361 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:54.821371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.822065 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.822070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:54.822178 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:54.822804 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:54.822830 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:54.826249 (+ 3419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":406,"threads_started":1}
W20260810 14:38:54.827117 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:54.817551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:54335 (local address 127.21.55.126:33539)
0810 14:38:54.817803 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:54.817813 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:54.818357 (+ 544us) server_negotiation.cc:407] Connection header received
0810 14:38:54.818512 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:54.818518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:54.818627 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:54.818745 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:54.819529 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.820293 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.821497 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:54.821914 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:54.822023 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:54.826881 (+ 4858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:54335: BlockingRecv error: recv got EOF from 127.21.55.67:54335 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 10 14:38:55 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:55 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:55.084656 4589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:55.072835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:42971)
0810 14:38:55.073776 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:55.073802 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:55.073982 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:55.074426 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:55.074432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:55.075117 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:55.075411 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:55.075428 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.076512 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.076516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:55.077527 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:55.077539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.078718 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.078724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:55.078848 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:55.079512 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:55.079531 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:55.084315 (+ 4784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":420,"threads_started":1}
W20260810 14:38:55.085181 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:55.073099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:42971 (local address 127.21.55.126:33539)
0810 14:38:55.073455 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:55.073465 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:55.073979 (+ 514us) server_negotiation.cc:407] Connection header received
0810 14:38:55.074112 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:55.074118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:55.074234 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:55.074380 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:55.075571 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.076378 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.077666 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.078470 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.078670 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:55.084649 (+ 5979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:42971: BlockingRecv error: recv got EOF from 127.21.55.65:42971 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
Aug 10 14:38:55 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:55 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:55.427290 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:55.413994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:52323)
0810 14:38:55.414622 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:55.414639 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:55.414808 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:55.415793 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:55.415798 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:55.416284 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:55.416534 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:55.416547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.418103 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.418110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:55.419737 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:55.419751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.420581 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.420587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:55.420713 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:55.421703 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:55.421743 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:55.426863 (+ 5120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":299,"threads_started":1}
W20260810 14:38:55.427914 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:55.414179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:52323 (local address 127.21.55.126:33539)
0810 14:38:55.414444 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:55.414450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:55.414730 (+ 280us) server_negotiation.cc:407] Connection header received
0810 14:38:55.415336 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:55.415349 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:55.415483 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:55.415769 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:55.416744 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.417872 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.419922 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.420328 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.420455 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:55.427399 (+ 6944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:52323: BlockingRecv error: recv got EOF from 127.21.55.66:52323 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 10 14:38:55 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:55 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:55.837217 4591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:55.827590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:50243)
0810 14:38:55.828346 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:55.828388 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:55.828539 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:55.828926 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:55.828931 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:55.829397 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:55.829563 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:55.829572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.830649 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.830656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:55.831603 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:55.831615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.832350 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.832357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:55.832522 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:55.833179 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:55.833197 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:55.836874 (+ 3677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":473,"threads_started":1}
W20260810 14:38:55.837371 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:55.827628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:50243 (local address 127.21.55.126:33539)
0810 14:38:55.827828 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:55.827835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:55.828480 (+ 645us) server_negotiation.cc:407] Connection header received
0810 14:38:55.828700 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:55.828705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:55.828783 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:55.828901 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:55.829722 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.830468 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.831748 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:55.832160 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:55.832297 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:55.837094 (+ 4797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:50243: BlockingRecv error: recv got EOF from 127.21.55.67:50243 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 14:38:56 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:56 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:56.097174 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:56.086069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:52815)
0810 14:38:56.086772 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:56.086796 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:56.086960 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:56.087364 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:56.087369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:56.087952 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:56.088179 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:56.088192 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.089701 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.089708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:56.091065 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:56.091080 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.092027 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.092042 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:56.092242 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:56.093013 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:56.093033 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:56.096923 (+ 3890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":315,"threads_started":1}
W20260810 14:38:56.097326 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:56.086125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:52815 (local address 127.21.55.126:33539)
0810 14:38:56.086367 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:56.086375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:56.086978 (+ 603us) server_negotiation.cc:407] Connection header received
0810 14:38:56.087086 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:56.087094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:56.087193 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:56.087284 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:56.088378 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.089450 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.091237 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.091665 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.091768 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:56.097080 (+ 5312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:52815: BlockingRecv error: recv got EOF from 127.21.55.65:52815 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 14:38:56 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:56 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:56.437822 4593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:56.428523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:34077)
0810 14:38:56.429112 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:56.429129 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:56.429284 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:56.429654 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:56.429662 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:56.430130 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:56.430278 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:56.430286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.431391 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.431398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:56.432347 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:56.432358 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.433096 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.433101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:56.433294 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:56.433892 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:56.433909 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:56.437574 (+ 3665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":358,"threads_started":1}
W20260810 14:38:56.438045 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:56.428606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:34077 (local address 127.21.55.126:33539)
0810 14:38:56.428891 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:56.428899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:56.429223 (+ 324us) server_negotiation.cc:407] Connection header received
0810 14:38:56.429417 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:56.429422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:56.429500 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:56.429609 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:56.430427 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.431249 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.432493 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.432931 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.433071 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:56.437748 (+ 4677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:34077: BlockingRecv error: recv got EOF from 127.21.55.66:34077 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 10 14:38:56 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:56 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:56.847405 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:56.838250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:39327)
0810 14:38:56.838840 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:56.838856 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:56.838964 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:56.839440 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:56.839443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:56.839872 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:56.840013 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:56.840020 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.840996 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.840999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:56.841948 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:56.841959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.842976 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.842981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:56.843107 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:56.843675 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:56.843692 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:56.847170 (+ 3478us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":338,"threads_started":1}
W20260810 14:38:56.847551 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:56.838374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:39327 (local address 127.21.55.126:33539)
0810 14:38:56.838637 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:56.838643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:56.838947 (+ 304us) server_negotiation.cc:407] Connection header received
0810 14:38:56.839206 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:56.839210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:56.839274 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:56.839363 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:56.840159 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.840867 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.842087 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:56.842809 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:56.842930 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:56.847367 (+ 4437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:39327: BlockingRecv error: recv got EOF from 127.21.55.67:39327 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 14:38:57 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:57 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:57.111603 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:57.098700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:43811)
0810 14:38:57.099540 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:57.099563 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:57.099802 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:57.100474 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:57.100480 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:57.101180 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:57.101596 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:57.101615 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.103132 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.103136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:57.104188 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:57.104364 (+ 176us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.105344 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.105350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:57.105521 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:57.106500 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:57.106549 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:57.111234 (+ 4685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":511,"threads_started":1}
W20260810 14:38:57.111771 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:57.098829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:43811 (local address 127.21.55.126:33539)
0810 14:38:57.099409 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:57.099419 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:57.099695 (+ 276us) server_negotiation.cc:407] Connection header received
0810 14:38:57.099984 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:57.099990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:57.100086 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:57.100209 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:57.101925 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.102969 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.104619 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.105099 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.105222 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:57.111456 (+ 6234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:43811: BlockingRecv error: recv got EOF from 127.21.55.65:43811 (error 108)
Metrics: {"server-negotiator.queue_time_us":374}
Aug 10 14:38:57 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:57 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:57.450112 4597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:57.438864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:35817)
0810 14:38:57.439362 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:57.439379 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:57.439515 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:57.440064 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:57.440068 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:57.440643 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:57.440865 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:57.440874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.442296 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.442304 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:57.443402 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:57.443414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.444552 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.444558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:57.444691 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:57.445439 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:57.445463 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:57.449854 (+ 4391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":273,"threads_started":1}
W20260810 14:38:57.450443 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:57.439031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:35817 (local address 127.21.55.126:33539)
0810 14:38:57.439262 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:57.439268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:57.439566 (+ 298us) server_negotiation.cc:407] Connection header received
0810 14:38:57.439689 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:57.439693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:57.439863 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:57.440006 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:57.441099 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.442117 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.443566 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.444326 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.444494 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:57.450106 (+ 5612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:35817: BlockingRecv error: recv got EOF from 127.21.55.66:35817 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 10 14:38:57 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:57 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:57.866178 4598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:57.849120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:56755)
0810 14:38:57.849784 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:57.849802 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:57.849999 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:57.850520 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:57.850562 (+ 42us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:57.851160 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:57.851395 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:57.851417 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.853016 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.853020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:57.854054 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:57.854066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.854896 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.854901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:57.855031 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:57.860415 (+ 5384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:57.860445 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:57.865832 (+ 5387us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":380,"threads_started":1}
W20260810 14:38:57.866365 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:57.849490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:56755 (local address 127.21.55.126:33539)
0810 14:38:57.849878 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:57.849884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:57.849904 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:38:57.850203 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:57.850211 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:57.850305 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:57.850450 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:57.851896 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.852867 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.854204 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:57.854736 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:57.854872 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:57.866087 (+ 11215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:56755: BlockingRecv error: recv got EOF from 127.21.55.67:56755 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
Aug 10 14:38:58 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:58 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:58.122414 4606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:58.112686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:55519)
0810 14:38:58.113297 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:58.113314 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:58.113484 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:58.113840 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:58.113846 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:58.114326 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:58.114513 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:58.114522 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.115574 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.115581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:58.116490 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:58.116502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.117286 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.117291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:58.117478 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:58.118137 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:58.118157 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:58.122048 (+ 3891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":311,"threads_started":1}
W20260810 14:38:58.122464 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:58.112862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:55519 (local address 127.21.55.126:33539)
0810 14:38:58.113132 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:58.113138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:58.113406 (+ 268us) server_negotiation.cc:407] Connection header received
0810 14:38:58.113613 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:58.113618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:58.113687 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:58.113794 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:58.114723 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.115437 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.116638 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.117124 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.117259 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:58.122237 (+ 4978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:55519: BlockingRecv error: recv got EOF from 127.21.55.65:55519 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 10 14:38:58 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:58 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:58.462980 4608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:58.451450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:40031)
0810 14:38:58.452254 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:58.452275 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:58.452433 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:58.452958 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:58.452963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:58.453456 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:58.453661 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:58.453671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.455301 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.455305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:58.456409 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:58.456421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.457524 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.457538 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:58.457679 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:58.458615 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:58.458647 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:58.462719 (+ 4072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":485,"threads_started":1}
W20260810 14:38:58.463541 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:58.451692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:40031 (local address 127.21.55.126:33539)
0810 14:38:58.451888 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:58.451894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:58.452441 (+ 547us) server_negotiation.cc:407] Connection header received
0810 14:38:58.452546 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:58.452551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:58.452627 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:58.452760 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:58.453904 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.455137 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.456592 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.457248 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.457357 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:58.462988 (+ 5631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:40031: BlockingRecv error: recv got EOF from 127.21.55.66:40031 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 14:38:58 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:58 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:58.880757 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:58.867718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:42147)
0810 14:38:58.868800 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:58.868915 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:58.869177 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:58.869696 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:58.869701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:58.870177 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:58.870377 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:58.870386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.871620 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.871624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:58.872554 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:58.872565 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.873505 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.873519 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:58.873731 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:58.874816 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:58.874837 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:58.880299 (+ 5462us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":772,"thread_start_us":629,"threads_started":1}
W20260810 14:38:58.880934 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:58.867764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:42147 (local address 127.21.55.126:33539)
0810 14:38:58.868005 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:58.868011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:58.869136 (+ 1125us) server_negotiation.cc:407] Connection header received
0810 14:38:58.869379 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:58.869384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:58.869471 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:58.869606 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:58.870587 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.871461 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.872732 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:58.873216 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:58.873353 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:58.880635 (+ 7282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:42147: BlockingRecv error: recv got EOF from 127.21.55.67:42147 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 10 14:38:59 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:59 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:59.137657 4611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:59.124010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:36259)
0810 14:38:59.124810 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:59.124832 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:59.125139 (+ 307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:59.125611 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:59.125616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:59.126235 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:59.126455 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:59.126492 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.127831 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.127841 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:59.129115 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:59.129132 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.130188 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.130195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:59.130442 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:59.131366 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:59.131390 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:59.137157 (+ 5767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":417,"threads_started":1}
W20260810 14:38:59.137924 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:59.124220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:36259 (local address 127.21.55.126:33539)
0810 14:38:59.124578 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:59.124587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:59.124989 (+ 402us) server_negotiation.cc:407] Connection header received
0810 14:38:59.125327 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:59.125333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:59.125431 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:59.125565 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:59.126738 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.127649 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.129316 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.129995 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.130161 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:59.137566 (+ 7405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:36259: BlockingRecv error: recv got EOF from 127.21.55.65:36259 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Aug 10 14:38:59 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:59 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:59.477231 4613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:59.464417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:46515)
0810 14:38:59.465141 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:59.465165 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:59.465381 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:59.465815 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:59.465821 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:59.466342 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:59.466623 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:59.466635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.467858 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.467867 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:59.468969 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:59.468982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.470063 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.470071 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:59.470369 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:59.471333 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:59.471356 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:59.476809 (+ 5453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":386,"threads_started":1}
W20260810 14:38:59.477399 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:59.464500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:46515 (local address 127.21.55.126:33539)
0810 14:38:59.464861 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:59.464872 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:59.465272 (+ 400us) server_negotiation.cc:407] Connection header received
0810 14:38:59.465551 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:59.465559 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:59.465648 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:59.465778 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:59.466805 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.467685 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.469142 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.469671 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.470057 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:59.477091 (+ 7034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:46515: BlockingRecv error: recv got EOF from 127.21.55.66:46515 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Aug 10 14:38:59 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:38:59 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:38:59.893666 4614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:38:59.882216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:57779)
0810 14:38:59.883042 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0810 14:38:59.883071 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 14:38:59.883227 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:38:59.883604 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:38:59.883607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:38:59.884022 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:38:59.884196 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:59.884204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.885433 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.885436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:59.886370 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:38:59.886382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.887290 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.887295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:38:59.887421 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:59.888232 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:38:59.888250 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:38:59.893215 (+ 4965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":419,"threads_started":1}
W20260810 14:38:59.893878 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:38:59.882280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:57779 (local address 127.21.55.126:33539)
0810 14:38:59.882580 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0810 14:38:59.882592 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:38:59.883186 (+ 594us) server_negotiation.cc:407] Connection header received
0810 14:38:59.883356 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:38:59.883361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:38:59.883445 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:38:59.883543 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:38:59.884339 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.885286 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.886586 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:38:59.887077 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:38:59.887233 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:38:59.893549 (+ 6316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:57779: BlockingRecv error: recv got EOF from 127.21.55.67:57779 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 14:39:00 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:39:00 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:39:00.151081 4615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:00.138892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:43859)
0810 14:39:00.139615 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:00.139646 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:00.139869 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:00.140308 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:00.140314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:00.140886 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:00.141067 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:00.141076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.142252 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.142257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:00.143354 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:00.143367 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.144417 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.144423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:00.144651 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:00.145660 (+ 1009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:39:00.145687 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:00.150659 (+ 4972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":388,"threads_started":1}
W20260810 14:39:00.151273 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:39:00.139195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:43859 (local address 127.21.55.126:33539)
0810 14:39:00.139537 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:00.139545 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:00.139765 (+ 220us) server_negotiation.cc:407] Connection header received
0810 14:39:00.140038 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:00.140044 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:00.140136 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:00.140265 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:00.141242 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.142083 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.143527 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.144015 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.144406 (+ 391us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:00.150972 (+ 6566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:43859: BlockingRecv error: recv got EOF from 127.21.55.65:43859 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Aug 10 14:39:00 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:39:00 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:39:00.491482 4616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:00.479382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:35141)
0810 14:39:00.480124 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:00.480143 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:00.480321 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:00.480679 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:00.480684 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:00.481182 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:00.481338 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:00.481346 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.482611 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.482614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:00.483708 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:00.483721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.484473 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.484479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:00.484796 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:00.485963 (+ 1167us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:39:00.485987 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:00.491115 (+ 5128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":494,"threads_started":1}
W20260810 14:39:00.491645 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:39:00.479582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:35141 (local address 127.21.55.126:33539)
0810 14:39:00.479872 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:00.479881 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:00.480244 (+ 363us) server_negotiation.cc:407] Connection header received
0810 14:39:00.480462 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:00.480467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:00.480535 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:00.480647 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:00.481488 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.482415 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.483870 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.484297 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.484442 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:00.491355 (+ 6913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:35141: BlockingRecv error: recv got EOF from 127.21.55.66:35141 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 10 14:39:00 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:39:00 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.67@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:39:00.907852 4617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:00.895101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.67:37505)
0810 14:39:00.895792 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:00.895809 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:00.895966 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:00.896371 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:00.896375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:00.896934 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:00.897130 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:00.897142 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.898388 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.898393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:00.899417 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:00.899429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.900682 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.900689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:00.900864 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:00.901777 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:39:00.901798 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:00.907360 (+ 5562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":464,"threads_started":1}
W20260810 14:39:00.908179 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:39:00.895346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:37505 (local address 127.21.55.126:33539)
0810 14:39:00.895558 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:00.895567 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:00.895891 (+ 324us) server_negotiation.cc:407] Connection header received
0810 14:39:00.896120 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:00.896125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:00.896203 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:00.896317 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:00.897274 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.898218 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.899662 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:00.900401 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:00.900571 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:00.907782 (+ 7211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.67:37505: BlockingRecv error: recv got EOF from 127.21.55.67:37505 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 14:39:01 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:39:01 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.65@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:39:01.163014 4618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:01.152848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.65:57323)
0810 14:39:01.153444 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:01.153461 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:01.153586 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:01.153880 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:01.153884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:01.154319 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:01.154505 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:01.154513 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.155607 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.155612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:01.156544 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:01.156556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.157376 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.157381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:01.157576 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:01.158341 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:39:01.158360 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:01.162625 (+ 4265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":321,"threads_started":1}
W20260810 14:39:01.163199 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:39:01.152938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:57323 (local address 127.21.55.126:33539)
0810 14:39:01.153219 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:01.153223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:01.153541 (+ 318us) server_negotiation.cc:407] Connection header received
0810 14:39:01.153681 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:01.153684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:01.153746 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:01.153831 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:01.154702 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.155445 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.156692 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.157225 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.157359 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:01.162862 (+ 5503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.65:57323: BlockingRecv error: recv got EOF from 127.21.55.65:57323 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Aug 10 14:39:01 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:39:01 dist-test-slave-h6n0 krb5kdc[2922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.55.66@KRBTEST.COM for oryx/127.21.55.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:39:01.504130 4619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:01.492950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33539 (local address 127.21.55.66:48371)
0810 14:39:01.493512 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:01.493528 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:01.493684 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:01.494056 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:01.494062 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:01.494583 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:01.494798 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:01.494807 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.495847 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.495852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:01.496818 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:01.496830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.498043 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.498050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:01.498289 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:01.499159 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:39:01.499181 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:01.503862 (+ 4681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.55.126:33539: Server oryx/127.21.55.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":313,"threads_started":1}
W20260810 14:39:01.504691 4544 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:39:01.493072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:48371 (local address 127.21.55.126:33539)
0810 14:39:01.493334 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:01.493341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:01.493617 (+ 276us) server_negotiation.cc:407] Connection header received
0810 14:39:01.493813 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:01.493821 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:01.493905 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:01.494011 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:01.494941 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.495693 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.497175 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.497774 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.498011 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:01.504423 (+ 6412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.55.66:48371: BlockingRecv error: recv got EOF from 127.21.55.66:48371 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 14:39:01.542377 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 3879
I20260810 14:39:01.556242 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:01.543735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.21.55.65:37277)
0810 14:39:01.544515 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:01.544522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:01.544552 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:39:01.544621 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:01.544628 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:01.544858 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:01.545031 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:01.546473 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.547669 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.549002 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.549595 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.549703 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:01.552950 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:01.552959 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:01.552981 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:39:01.553020 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:01.555707 (+ 2687us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.55.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 14:39:01.555996 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60076: Request ticket server kudu/127.21.55.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":423,"threads_started":1}
W20260810 14:39:01.556329 4621 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60076: Request ticket server kudu/127.21.55.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 14:39:01.556777 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 3879: Not authorized: Client connection negotiation failed: client connection to 127.21.55.65:37277: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.55.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 14:39:01.557008 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 3879
I20260810 14:39:01.594240 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 4120
I20260810 14:39:01.607376 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:01.595348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57182 (local address 127.21.55.66:35785)
0810 14:39:01.596004 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:01.596014 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:01.596048 (+ 34us) server_negotiation.cc:407] Connection header received
0810 14:39:01.596127 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:01.596136 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:01.596437 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:01.596663 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:01.598036 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.599528 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.600762 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.601329 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.601432 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:01.604075 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:01.604084 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:01.604097 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:39:01.604135 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:01.606819 (+ 2684us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.55.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 14:39:01.607113 (+ 294us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57182: Request ticket server kudu/127.21.55.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":303,"threads_started":1}
W20260810 14:39:01.607487 4622 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57182: Request ticket server kudu/127.21.55.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 14:39:01.607724 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 4120: Not authorized: Client connection negotiation failed: client connection to 127.21.55.66:35785: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.55.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 14:39:01.607895 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 4120
I20260810 14:39:01.631331 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 4329
I20260810 14:39:01.645792 4626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:01.632684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57038 (local address 127.21.55.67:33289)
0810 14:39:01.633602 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:01.633610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:01.633639 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:39:01.633713 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:01.633720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:01.633973 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:01.634223 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:01.635725 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.636748 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.637954 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:01.638500 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:01.638687 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:01.641469 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:01.641481 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:01.641501 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:39:01.641564 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:01.645182 (+ 3618us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.55.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 14:39:01.645497 (+ 315us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57038: Request ticket server kudu/127.21.55.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":557,"threads_started":1}
W20260810 14:39:01.645929 4626 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57038: Request ticket server kudu/127.21.55.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 14:39:01.646381 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 4329: Not authorized: Client connection negotiation failed: client connection to 127.21.55.67:33289: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.55.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 14:39:01.646603 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 4329
I20260810 14:39:01.673883 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 3759
I20260810 14:39:01.675299 3861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:01.796155 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 3759
2026-08-10T14:39:01Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22188 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T14:39:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:39:01Z Disabled control of system clock
I20260810 14:39:01.893707 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:39295
--webserver_interface=127.21.55.126
--webserver_port=0
--builtin_ntp_servers=127.21.55.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:39295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 14:39:02.241693 4636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:02.241976 4636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:02.242053 4636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:02.249229 4636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:39:02.249339 4636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:02.249378 4636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:02.249429 4636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:39:02.249480 4636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:39:02.259845 4636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:39295
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:39295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:02.261130 4636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:02.262984 4636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:02.278910 4646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:02.278910 4647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:02.280066 4636 server_base.cc:1034] running on GCE node
W20260810 14:39:02.280332 4657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:02.281932 4636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:02.283473 4636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:02.284725 4636 hybrid_clock.cc:648] HybridClock initialized: now 1786372742284682 us; error 61 us; skew 500 ppm
I20260810 14:39:02.285243 4636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:39:02.287797 4636 webserver.cc:460] Webserver started at http://127.21.55.126:33401/ using document root <none> and password file <none>
I20260810 14:39:02.288535 4636 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:02.288698 4636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:02.289034 4636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:02.291998 4636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "8ca4221159af4882a681dc1cb8f2afaa"
format_stamp: "Formatted at 2026-08-10 14:39:02 on dist-test-slave-h6n0"
server_key: "07f44a87af414d853849d754dd530297"
server_key_iv: "4d0799cf460a5574fad6c1e08e09df2a"
server_key_version: "encryptionkey@0"
I20260810 14:39:02.292801 4636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "8ca4221159af4882a681dc1cb8f2afaa"
format_stamp: "Formatted at 2026-08-10 14:39:02 on dist-test-slave-h6n0"
server_key: "07f44a87af414d853849d754dd530297"
server_key_iv: "4d0799cf460a5574fad6c1e08e09df2a"
server_key_version: "encryptionkey@0"
I20260810 14:39:02.298029 4636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 14:39:02.301680 4670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:02.303139 4636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:39:02.303345 4636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "8ca4221159af4882a681dc1cb8f2afaa"
format_stamp: "Formatted at 2026-08-10 14:39:02 on dist-test-slave-h6n0"
server_key: "07f44a87af414d853849d754dd530297"
server_key_iv: "4d0799cf460a5574fad6c1e08e09df2a"
server_key_version: "encryptionkey@0"
I20260810 14:39:02.303574 4636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:02.321017 4636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:02.321828 4636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:02.322139 4636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:02.344439 4636 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:39295
I20260810 14:39:02.344477 4746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:39295 every 8 connection(s)
I20260810 14:39:02.345765 4636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:39:02.350497 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 4636
I20260810 14:39:02.350841 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:39:02.351467 4747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:39:02.351747 21725 external_mini_cluster.cc:1428] Setting key 2dde60ad856b67af1263fd7ef77928bd
I20260810 14:39:02.371656 4747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa: Bootstrap starting.
I20260810 14:39:02.376021 4751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:02.354845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53906 (local address 127.21.55.126:39295)
0810 14:39:02.356501 (+ 1656us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:02.356521 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:02.356604 (+ 83us) server_negotiation.cc:407] Connection header received
0810 14:39:02.360515 (+ 3911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:02.360564 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:02.361560 (+ 996us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:02.362219 (+ 659us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:02.363185 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:02.365614 (+ 2429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:02.367403 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:02.368864 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:02.369098 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:02.371060 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:02.371075 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:02.371100 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:02.372504 (+ 1404us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:02.372667 (+ 163us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:02.372716 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:02.372887 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:02.373245 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 14:39:02.373707 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":975,"thread_start_us":601,"threads_started":1}
I20260810 14:39:02.378024 4747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa: Neither blocks nor log segments found. Creating new log.
I20260810 14:39:02.379665 4747 log.cc:824] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa: Log is configured to *not* fsync() on all Append() calls
I20260810 14:39:02.382751 4747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa: No bootstrap required, opened a new log
I20260810 14:39:02.390017 4747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4221159af4882a681dc1cb8f2afaa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39295 } }
I20260810 14:39:02.390399 4747 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:39:02.390522 4747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ca4221159af4882a681dc1cb8f2afaa, State: Initialized, Role: FOLLOWER
I20260810 14:39:02.391182 4747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4221159af4882a681dc1cb8f2afaa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39295 } }
I20260810 14:39:02.391424 4747 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:39:02.391582 4747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:39:02.391773 4747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:39:02.394141 4747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4221159af4882a681dc1cb8f2afaa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39295 } }
I20260810 14:39:02.394742 4747 leader_election.cc:302] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ca4221159af4882a681dc1cb8f2afaa; no voters:
I20260810 14:39:02.395340 4747 leader_election.cc:288] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:39:02.395632 4756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:39:02.396564 4756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [term 1 LEADER]: Becoming Leader. State: Replica: 8ca4221159af4882a681dc1cb8f2afaa, State: Running, Role: LEADER
I20260810 14:39:02.397543 4756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4221159af4882a681dc1cb8f2afaa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39295 } }
I20260810 14:39:02.399099 4747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:39:02.400673 4758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ca4221159af4882a681dc1cb8f2afaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4221159af4882a681dc1cb8f2afaa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39295 } } }
I20260810 14:39:02.401078 4758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [sys.catalog]: This master's current role is: LEADER
I20260810 14:39:02.401885 4757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ca4221159af4882a681dc1cb8f2afaa. Latest consensus state: current_term: 1 leader_uuid: "8ca4221159af4882a681dc1cb8f2afaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4221159af4882a681dc1cb8f2afaa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39295 } } }
I20260810 14:39:02.402563 4757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa [sys.catalog]: This master's current role is: LEADER
I20260810 14:39:02.403692 4762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:39:02.411746 4762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:39:02.423466 4762 catalog_manager.cc:1347] Generated new cluster ID: bbb2231245f6499f97e928070deb5948
I20260810 14:39:02.423610 4762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:39:02.447213 4762 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:39:02.449072 4762 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:39:02.466228 4762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ca4221159af4882a681dc1cb8f2afaa: Generated new TSK 0
I20260810 14:39:02.467289 4762 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:39:02.481892 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:39295
--builtin_ntp_servers=127.21.55.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 14:39:02.854027 4788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:02.854306 4788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:02.854444 4788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:02.861754 4788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:02.861846 4788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:02.862006 4788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:39:02.871209 4788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:39295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:02.872507 4788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:02.874248 4788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:02.888224 4800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:02.888154 4794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:02.888124 4804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:02.889343 4788 server_base.cc:1034] running on GCE node
I20260810 14:39:02.889914 4788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:02.890966 4788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:02.892182 4788 hybrid_clock.cc:648] HybridClock initialized: now 1786372742892135 us; error 41 us; skew 500 ppm
I20260810 14:39:02.892514 4788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:39:02.900014 4788 webserver.cc:460] Webserver started at http://127.21.55.65:35761/ using document root <none> and password file <none>
I20260810 14:39:02.901019 4788 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:02.901376 4788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:02.901787 4788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:02.904757 4788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "4217efcf5d5348bab4b7e7a322e47499"
format_stamp: "Formatted at 2026-08-10 14:39:02 on dist-test-slave-h6n0"
server_key: "20b407c06a5ce3cfc3fa87ddc9af6b9b"
server_key_iv: "5a2114901392d2ebdfd31c371f924ea8"
server_key_version: "encryptionkey@0"
I20260810 14:39:02.905568 4788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "4217efcf5d5348bab4b7e7a322e47499"
format_stamp: "Formatted at 2026-08-10 14:39:02 on dist-test-slave-h6n0"
server_key: "20b407c06a5ce3cfc3fa87ddc9af6b9b"
server_key_iv: "5a2114901392d2ebdfd31c371f924ea8"
server_key_version: "encryptionkey@0"
I20260810 14:39:02.911857 4788 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 14:39:02.916074 4811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:02.917743 4788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260810 14:39:02.917951 4788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "4217efcf5d5348bab4b7e7a322e47499"
format_stamp: "Formatted at 2026-08-10 14:39:02 on dist-test-slave-h6n0"
server_key: "20b407c06a5ce3cfc3fa87ddc9af6b9b"
server_key_iv: "5a2114901392d2ebdfd31c371f924ea8"
server_key_version: "encryptionkey@0"
I20260810 14:39:02.918210 4788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:02.940212 4788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:02.941061 4788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:02.941385 4788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:02.942432 4788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:39:02.943934 4788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:39:02.944077 4788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:02.944205 4788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:39:02.944298 4788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:02.987924 4788 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:37887
I20260810 14:39:02.988035 5006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:37887 every 8 connection(s)
I20260810 14:39:02.989481 4788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:39:02.993475 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 4788
I20260810 14:39:02.993738 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:39:02.994716 21725 external_mini_cluster.cc:1428] Setting key 0a9e2dea4076c9e5e9d0adf7e38541b1
I20260810 14:39:02.999023 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:39295
--builtin_ntp_servers=127.21.55.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 14:39:03.006510 5009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:02.993695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:36869 (local address 127.21.55.126:39295)
0810 14:39:02.994585 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:02.994672 (+ 87us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:02.996331 (+ 1659us) server_negotiation.cc:407] Connection header received
0810 14:39:02.998770 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:02.998776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:02.998868 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:02.999003 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:03.000290 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:03.001100 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:03.002731 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:03.003444 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:03.003694 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:03.005558 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:03.005563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:03.005566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:03.005584 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:03.005644 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:03.005658 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:03.005764 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:03.006006 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 14:39:03.006119 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":414,"threads_started":1}
I20260810 14:39:03.007927 5010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:02.994699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:39295 (local address 127.21.55.65:36869)
0810 14:39:02.996153 (+ 1454us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:02.996198 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:02.998416 (+ 2218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:02.999278 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:02.999289 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:02.999419 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:03.000115 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:03.000134 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:03.001264 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:03.001276 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:03.002506 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:03.002518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:03.003920 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:03.003929 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:03.004234 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:03.005164 (+ 930us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:39:03.005168 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:03.005222 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:39:03.005315 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:39:03.005766 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:39:03.005780 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:39:03.005791 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 14:39:03.005991 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 14:39:03.006415 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":494,"threads_started":1}
I20260810 14:39:03.009608 5007 heartbeater.cc:347] Connected to a master server at 127.21.55.126:39295
I20260810 14:39:03.009934 5007 heartbeater.cc:464] Registering TS with master...
I20260810 14:39:03.010833 5007 heartbeater.cc:511] Master 127.21.55.126:39295 requested a full tablet report, sending...
I20260810 14:39:03.013144 4692 ts_manager.cc:194] Registered new tserver with Master: 4217efcf5d5348bab4b7e7a322e47499 (127.21.55.65:37887)
I20260810 14:39:03.014688 4692 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.65:36869
W20260810 14:39:03.368027 5012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:03.368427 5012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:03.368616 5012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:03.376128 5012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:03.376235 5012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:03.376413 5012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:39:03.385618 5012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:39295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:03.387033 5012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:03.389496 5012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:03.405098 5025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:03.405179 5027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:03.405141 5030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:03.406239 5012 server_base.cc:1034] running on GCE node
I20260810 14:39:03.407197 5012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:03.408320 5012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:03.409548 5012 hybrid_clock.cc:648] HybridClock initialized: now 1786372743409492 us; error 44 us; skew 500 ppm
I20260810 14:39:03.409904 5012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:39:03.412233 5012 webserver.cc:460] Webserver started at http://127.21.55.66:42021/ using document root <none> and password file <none>
I20260810 14:39:03.413069 5012 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:03.413166 5012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:03.413532 5012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:03.416392 5012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "567f62b317a743f798d6dd39b5701f53"
format_stamp: "Formatted at 2026-08-10 14:39:03 on dist-test-slave-h6n0"
server_key: "6ffb51125c08e75022d0551c7655ec2a"
server_key_iv: "f5a33f1f4942fb73514fbe1204a0937f"
server_key_version: "encryptionkey@0"
I20260810 14:39:03.417201 5012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "567f62b317a743f798d6dd39b5701f53"
format_stamp: "Formatted at 2026-08-10 14:39:03 on dist-test-slave-h6n0"
server_key: "6ffb51125c08e75022d0551c7655ec2a"
server_key_iv: "f5a33f1f4942fb73514fbe1204a0937f"
server_key_version: "encryptionkey@0"
I20260810 14:39:03.422453 5012 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:39:03.426255 5039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:03.427979 5012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 14:39:03.428211 5012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "567f62b317a743f798d6dd39b5701f53"
format_stamp: "Formatted at 2026-08-10 14:39:03 on dist-test-slave-h6n0"
server_key: "6ffb51125c08e75022d0551c7655ec2a"
server_key_iv: "f5a33f1f4942fb73514fbe1204a0937f"
server_key_version: "encryptionkey@0"
I20260810 14:39:03.428467 5012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:03.445189 5012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:03.446072 5012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:03.446422 5012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:03.447497 5012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:39:03.449102 5012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:39:03.449240 5012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:03.449393 5012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:39:03.449497 5012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:03.491149 5012 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:39589
I20260810 14:39:03.491233 5228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:39589 every 8 connection(s)
I20260810 14:39:03.493117 5012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:39:03.502362 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 5012
I20260810 14:39:03.502595 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:39:03.503080 21725 external_mini_cluster.cc:1428] Setting key 45d17b387622cd7a08fa7f365c7fc600
I20260810 14:39:03.508481 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:39295
--builtin_ntp_servers=127.21.55.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 14:39:03.511406 5009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:03.497021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:56673 (local address 127.21.55.126:39295)
0810 14:39:03.497274 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:03.497279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:03.498376 (+ 1097us) server_negotiation.cc:407] Connection header received
0810 14:39:03.500652 (+ 2276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:03.500658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:03.500775 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:03.500915 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:03.502798 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:03.503692 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:03.506963 (+ 3271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:03.507744 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:03.507832 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:03.510106 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:03.510112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:03.510116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:03.510137 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:03.510173 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:03.510187 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:03.510301 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:03.510608 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 14:39:03.510725 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 14:39:03.513015 5235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:03.497202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:39295 (local address 127.21.55.66:56673)
0810 14:39:03.498193 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:03.498239 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:03.500355 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:03.501453 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:03.501465 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:03.501598 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:03.502384 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:03.502405 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:03.503906 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:03.503915 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:03.506635 (+ 2720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:03.506665 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:03.508222 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:03.508227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:03.508615 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:03.509669 (+ 1054us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:39:03.509673 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:03.509719 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:39:03.509812 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:39:03.510305 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:39:03.510320 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:39:03.510332 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 14:39:03.510585 (+ 253us) client_negotiation.cc:240] Negotiation successful
0810 14:39:03.511385 (+ 800us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":330,"threads_started":1}
I20260810 14:39:03.515086 5229 heartbeater.cc:347] Connected to a master server at 127.21.55.126:39295
I20260810 14:39:03.515488 5229 heartbeater.cc:464] Registering TS with master...
I20260810 14:39:03.516705 5229 heartbeater.cc:511] Master 127.21.55.126:39295 requested a full tablet report, sending...
I20260810 14:39:03.518514 4692 ts_manager.cc:194] Registered new tserver with Master: 567f62b317a743f798d6dd39b5701f53 (127.21.55.66:39589)
I20260810 14:39:03.519749 4692 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.66:56673
W20260810 14:39:03.883291 5236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:03.883648 5236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:03.883821 5236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:03.891376 5236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:03.891534 5236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:03.891718 5236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:39:03.901892 5236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:39295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:03.903267 5236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:03.905107 5236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:03.919271 5244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:03.919488 5250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:03.919710 5245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:03.920608 5236 server_base.cc:1034] running on GCE node
I20260810 14:39:03.921299 5236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:03.922649 5236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:03.924026 5236 hybrid_clock.cc:648] HybridClock initialized: now 1786372743923951 us; error 79 us; skew 500 ppm
I20260810 14:39:03.924422 5236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:39:03.929155 5236 webserver.cc:460] Webserver started at http://127.21.55.67:34737/ using document root <none> and password file <none>
I20260810 14:39:03.930061 5236 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:03.930218 5236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:03.930719 5236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:03.934108 5236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "6aaa79c93d3e4044a1ef354ecdd26582"
format_stamp: "Formatted at 2026-08-10 14:39:03 on dist-test-slave-h6n0"
server_key: "7977754cbb9906bc4d6b4f06904c1822"
server_key_iv: "2c269e55de143e8047cf629b8f80b4b4"
server_key_version: "encryptionkey@0"
I20260810 14:39:03.935287 5236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "6aaa79c93d3e4044a1ef354ecdd26582"
format_stamp: "Formatted at 2026-08-10 14:39:03 on dist-test-slave-h6n0"
server_key: "7977754cbb9906bc4d6b4f06904c1822"
server_key_iv: "2c269e55de143e8047cf629b8f80b4b4"
server_key_version: "encryptionkey@0"
I20260810 14:39:03.941895 5236 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 14:39:03.947922 5264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:03.950022 5236 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260810 14:39:03.950313 5236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "6aaa79c93d3e4044a1ef354ecdd26582"
format_stamp: "Formatted at 2026-08-10 14:39:03 on dist-test-slave-h6n0"
server_key: "7977754cbb9906bc4d6b4f06904c1822"
server_key_iv: "2c269e55de143e8047cf629b8f80b4b4"
server_key_version: "encryptionkey@0"
I20260810 14:39:03.950665 5236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:03.964452 5236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:03.965348 5236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:03.965685 5236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:03.967633 5236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:39:03.969194 5236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:39:03.969321 5236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:03.969460 5236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:39:03.969565 5236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:04.014114 5236 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:38123
I20260810 14:39:04.014323 5443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:38123 every 8 connection(s)
I20260810 14:39:04.015666 5236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:39:04.017612 5007 heartbeater.cc:503] Master 127.21.55.126:39295 was elected leader, sending a full tablet report...
I20260810 14:39:04.022943 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 5236
I20260810 14:39:04.023164 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:39:04.023746 21725 external_mini_cluster.cc:1428] Setting key 535d5f6691b32c966741652cba663208
I20260810 14:39:04.030987 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:04.019498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:53025 (local address 127.21.55.126:39295)
0810 14:39:04.020066 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:04.020072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:04.021339 (+ 1267us) server_negotiation.cc:407] Connection header received
0810 14:39:04.023651 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:04.023655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:04.023718 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:04.023818 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:04.025139 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:04.026372 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:04.027995 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:04.028414 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:04.028549 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:04.029986 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:04.029992 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:04.029995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:04.030012 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:04.030059 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:04.030071 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:04.030175 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:04.030428 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 14:39:04.030560 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":347,"threads_started":1}
I20260810 14:39:04.032083 5447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:04.020237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:39295 (local address 127.21.55.67:53025)
0810 14:39:04.021165 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:04.021215 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:04.023322 (+ 2107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:04.024128 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:04.024141 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:04.024267 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:04.024938 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:04.024962 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:04.026585 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:04.026595 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:04.027835 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:04.027847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:04.028667 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:04.028675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:04.028904 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:04.029655 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:39:04.029658 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:04.029703 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:39:04.029779 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:39:04.030176 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:39:04.030190 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:39:04.030200 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:39:04.030394 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 14:39:04.030830 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":285,"threads_started":1}
I20260810 14:39:04.033605 5444 heartbeater.cc:347] Connected to a master server at 127.21.55.126:39295
I20260810 14:39:04.033962 5444 heartbeater.cc:464] Registering TS with master...
I20260810 14:39:04.034897 5444 heartbeater.cc:511] Master 127.21.55.126:39295 requested a full tablet report, sending...
I20260810 14:39:04.036664 4692 ts_manager.cc:194] Registered new tserver with Master: 6aaa79c93d3e4044a1ef354ecdd26582 (127.21.55.67:38123)
I20260810 14:39:04.037542 4692 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.67:53025
I20260810 14:39:04.040351 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 14:39:04.047644 5446 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:39:04.045862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53922 (local address 127.21.55.126:39295)
0810 14:39:04.046167 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:04.046176 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:04.046564 (+ 388us) server_negotiation.cc:407] Connection header received
0810 14:39:04.046808 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:04.046812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:04.046872 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:04.046976 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:04.047458 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53922: BlockingRecv error: recv got EOF from 127.0.0.1:53922 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:39:04.048864 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 4788
I20260810 14:39:04.057971 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:04.049792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42060 (local address 127.21.55.65:37887)
0810 14:39:04.050380 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:04.050387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:04.050413 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:39:04.050470 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:04.050476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:04.050991 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:04.051160 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:04.051797 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:04.052888 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:04.054301 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:04.055046 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:04.055165 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:04.056561 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:04.056571 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:04.056584 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:04.057176 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:04.057229 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:04.057252 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:04.057371 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:04.057528 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 14:39:04.057639 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":279,"threads_started":1}
I20260810 14:39:04.059628 5001 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:04.149865 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 4788
I20260810 14:39:04.169770 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 5012
I20260810 14:39:04.180071 5465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:04.170991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53148 (local address 127.21.55.66:39589)
0810 14:39:04.171613 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:04.171620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:04.171648 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:39:04.171718 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:04.171724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:04.171956 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:04.172101 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:04.172711 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:04.174157 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:04.175600 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:04.176196 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:04.176304 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:04.177585 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:04.177599 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:04.177620 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:04.178789 (+ 1169us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:04.178882 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:04.178914 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:04.179055 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:04.179307 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 14:39:04.179600 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":337,"threads_started":1}
I20260810 14:39:04.181510 5223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:04.268723 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 5012
I20260810 14:39:04.301882 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 5236
I20260810 14:39:04.311434 5467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:04.303050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47574 (local address 127.21.55.67:38123)
0810 14:39:04.303678 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:04.303686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:04.303715 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:39:04.303776 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:04.303783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:04.303996 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:04.304131 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:04.304889 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:04.305960 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:04.307875 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:04.308628 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:04.308739 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:04.309903 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:04.309913 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:04.309926 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:04.310610 (+ 684us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:04.310658 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:04.310678 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:04.310782 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:04.310991 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 14:39:04.311124 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":320,"threads_started":1}
I20260810 14:39:04.312695 5439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:04.400379 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 5236
I20260810 14:39:04.425103 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 4636
I20260810 14:39:04.426774 4741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:04.544680 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 4636
2026-08-10T14:39:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2749 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T14:39:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:39:04Z Disabled control of system clock
I20260810 14:39:04.645128 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:45539
--webserver_interface=127.21.55.126
--webserver_port=0
--builtin_ntp_servers=127.21.55.84:43125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:45539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 14:39:05.013401 5477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:05.013724 5477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:05.013893 5477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:05.021754 5477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:39:05.021863 5477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:05.021903 5477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:05.021991 5477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:39:05.022068 5477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:39:05.031323 5477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:45539
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:45539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:05.033013 5477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:05.035023 5477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:05.048638 5487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:05.048635 5486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:05.049647 5492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:05.049829 5477 server_base.cc:1034] running on GCE node
I20260810 14:39:05.051066 5477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:05.052502 5477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:05.053761 5477 hybrid_clock.cc:648] HybridClock initialized: now 1786372745053708 us; error 45 us; skew 500 ppm
I20260810 14:39:05.054262 5477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:39:05.061281 5477 webserver.cc:460] Webserver started at http://127.21.55.126:44905/ using document root <none> and password file <none>
I20260810 14:39:05.062106 5477 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:05.062268 5477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:05.062759 5477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:05.067034 5477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "100fb4aa55774c38940be305aeb4f4da"
format_stamp: "Formatted at 2026-08-10 14:39:05 on dist-test-slave-h6n0"
server_key: "297bc458e651c842588b90363fe45860"
server_key_iv: "9e3887f2c9719653588c1d2037ab7ead"
server_key_version: "encryptionkey@0"
I20260810 14:39:05.068161 5477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "100fb4aa55774c38940be305aeb4f4da"
format_stamp: "Formatted at 2026-08-10 14:39:05 on dist-test-slave-h6n0"
server_key: "297bc458e651c842588b90363fe45860"
server_key_iv: "9e3887f2c9719653588c1d2037ab7ead"
server_key_version: "encryptionkey@0"
I20260810 14:39:05.074640 5477 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 14:39:05.078259 5502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:05.079669 5477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 14:39:05.079869 5477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "100fb4aa55774c38940be305aeb4f4da"
format_stamp: "Formatted at 2026-08-10 14:39:05 on dist-test-slave-h6n0"
server_key: "297bc458e651c842588b90363fe45860"
server_key_iv: "9e3887f2c9719653588c1d2037ab7ead"
server_key_version: "encryptionkey@0"
I20260810 14:39:05.080118 5477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:05.085868 5477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:05.086699 5477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:05.087002 5477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:05.108479 5477 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:45539
I20260810 14:39:05.108518 5589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:45539 every 8 connection(s)
I20260810 14:39:05.109916 5477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:39:05.113969 5591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:39:05.115345 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 5477
I20260810 14:39:05.115612 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:39:05.116281 21725 external_mini_cluster.cc:1428] Setting key 0351ee72cc7be26872a1ba1c15ce724a
I20260810 14:39:05.124974 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:05.118322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59872 (local address 127.21.55.126:45539)
0810 14:39:05.119453 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:05.119472 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:05.119528 (+ 56us) server_negotiation.cc:407] Connection header received
0810 14:39:05.121155 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:05.121181 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:05.121686 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:05.122111 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:05.123073 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:05.123083 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:05.123097 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:05.123113 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:05.123179 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:05.123199 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:05.123404 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:05.123554 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 14:39:05.123782 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":408,"threads_started":1}
I20260810 14:39:05.128983 5591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da: Bootstrap starting.
I20260810 14:39:05.132292 5591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da: Neither blocks nor log segments found. Creating new log.
I20260810 14:39:05.133291 5591 log.cc:824] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da: Log is configured to *not* fsync() on all Append() calls
I20260810 14:39:05.136194 5591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da: No bootstrap required, opened a new log
I20260810 14:39:05.144490 5591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100fb4aa55774c38940be305aeb4f4da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45539 } }
I20260810 14:39:05.144909 5591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:39:05.145027 5591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 100fb4aa55774c38940be305aeb4f4da, State: Initialized, Role: FOLLOWER
I20260810 14:39:05.146260 5591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100fb4aa55774c38940be305aeb4f4da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45539 } }
I20260810 14:39:05.146773 5591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:39:05.146950 5591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:39:05.147264 5591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:39:05.150658 5591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100fb4aa55774c38940be305aeb4f4da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45539 } }
I20260810 14:39:05.151243 5591 leader_election.cc:302] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 100fb4aa55774c38940be305aeb4f4da; no voters:
I20260810 14:39:05.151894 5591 leader_election.cc:288] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:39:05.152009 5600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:39:05.152802 5600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [term 1 LEADER]: Becoming Leader. State: Replica: 100fb4aa55774c38940be305aeb4f4da, State: Running, Role: LEADER
I20260810 14:39:05.153355 5600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100fb4aa55774c38940be305aeb4f4da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45539 } }
I20260810 14:39:05.154198 5591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:39:05.156757 5601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "100fb4aa55774c38940be305aeb4f4da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100fb4aa55774c38940be305aeb4f4da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45539 } } }
I20260810 14:39:05.156843 5602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 100fb4aa55774c38940be305aeb4f4da. Latest consensus state: current_term: 1 leader_uuid: "100fb4aa55774c38940be305aeb4f4da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100fb4aa55774c38940be305aeb4f4da" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 45539 } } }
I20260810 14:39:05.157248 5601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [sys.catalog]: This master's current role is: LEADER
I20260810 14:39:05.157281 5602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da [sys.catalog]: This master's current role is: LEADER
I20260810 14:39:05.159403 5615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:39:05.164589 5615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:39:05.173452 5615 catalog_manager.cc:1347] Generated new cluster ID: e72989e54a8e4e27af265ff1e90a62b3
I20260810 14:39:05.173578 5615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:39:05.194983 5615 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:39:05.195474 5615 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:39:05.214336 5615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 100fb4aa55774c38940be305aeb4f4da: Generated new TSK 0
I20260810 14:39:05.215428 5615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:39:05.228802 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45539
--builtin_ntp_servers=127.21.55.84:43125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 14:39:05.654278 5635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:05.654644 5635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:05.654789 5635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:05.662762 5635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:05.662878 5635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:05.663044 5635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:39:05.672523 5635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:05.674324 5635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:05.676232 5635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:05.690989 5644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:05.691107 5647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:05.691148 5652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:05.692113 5635 server_base.cc:1034] running on GCE node
I20260810 14:39:05.692735 5635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:05.693679 5635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:05.695174 5635 hybrid_clock.cc:648] HybridClock initialized: now 1786372745695083 us; error 137 us; skew 500 ppm
I20260810 14:39:05.695837 5635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:39:05.698650 5635 webserver.cc:460] Webserver started at http://127.21.55.65:45717/ using document root <none> and password file <none>
I20260810 14:39:05.699376 5635 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:05.699524 5635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:05.699877 5635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:05.702840 5635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "738b87aeece74fe58502a6d8398bdd9c"
format_stamp: "Formatted at 2026-08-10 14:39:05 on dist-test-slave-h6n0"
server_key: "30e49bcb220e579b849e54998c681a5a"
server_key_iv: "57e6a50f99fab8a75e434abedc559a33"
server_key_version: "encryptionkey@0"
I20260810 14:39:05.703663 5635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "738b87aeece74fe58502a6d8398bdd9c"
format_stamp: "Formatted at 2026-08-10 14:39:05 on dist-test-slave-h6n0"
server_key: "30e49bcb220e579b849e54998c681a5a"
server_key_iv: "57e6a50f99fab8a75e434abedc559a33"
server_key_version: "encryptionkey@0"
I20260810 14:39:05.708971 5635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 14:39:05.713425 5663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:05.714977 5635 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:39:05.715150 5635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "738b87aeece74fe58502a6d8398bdd9c"
format_stamp: "Formatted at 2026-08-10 14:39:05 on dist-test-slave-h6n0"
server_key: "30e49bcb220e579b849e54998c681a5a"
server_key_iv: "57e6a50f99fab8a75e434abedc559a33"
server_key_version: "encryptionkey@0"
I20260810 14:39:05.715386 5635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:05.720979 5635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:05.721766 5635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:05.722046 5635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:05.723047 5635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:39:05.724536 5635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:39:05.724632 5635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:05.724707 5635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:39:05.724756 5635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:05.763944 5635 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:39761
I20260810 14:39:05.764022 5866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:39761 every 8 connection(s)
I20260810 14:39:05.765282 5635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:39:05.765861 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 5635
I20260810 14:39:05.766108 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:39:05.766933 21725 external_mini_cluster.cc:1428] Setting key 1aceb1e108247db1aeb47eb3a6423070
I20260810 14:39:05.770500 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45539
--builtin_ntp_servers=127.21.55.84:43125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 14:39:05.774930 5873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:05.768179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:56663 (local address 127.21.55.126:45539)
0810 14:39:05.768634 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:05.768638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:05.770275 (+ 1637us) server_negotiation.cc:407] Connection header received
0810 14:39:05.772287 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:05.772293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:05.772385 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:05.772519 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:05.774007 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:05.774013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:05.774018 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:05.774040 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:05.774099 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:05.774112 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:05.774264 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:05.774480 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 14:39:05.774579 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":249,"threads_started":1}
I20260810 14:39:05.775923 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:05.768883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45539 (local address 127.21.55.65:56663)
0810 14:39:05.770112 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:05.770151 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:05.771998 (+ 1847us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:05.772764 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:05.772772 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:05.772869 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:05.773668 (+ 799us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:39:05.773670 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:05.773718 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:39:05.773806 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:39:05.774223 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:39:05.774237 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:39:05.774247 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:39:05.774610 (+ 363us) client_negotiation.cc:240] Negotiation successful
0810 14:39:05.774875 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":488,"threads_started":1}
I20260810 14:39:05.780229 5867 heartbeater.cc:347] Connected to a master server at 127.21.55.126:45539
I20260810 14:39:05.780612 5867 heartbeater.cc:464] Registering TS with master...
I20260810 14:39:05.781455 5867 heartbeater.cc:511] Master 127.21.55.126:45539 requested a full tablet report, sending...
I20260810 14:39:05.784437 5537 ts_manager.cc:194] Registered new tserver with Master: 738b87aeece74fe58502a6d8398bdd9c (127.21.55.65:39761)
W20260810 14:39:06.124595 5876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:06.125016 5876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:06.125217 5876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:06.133733 5876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:06.133919 5876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:06.134140 5876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:39:06.145044 5876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:06.146867 5876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:06.149488 5876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:06.166493 5883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:06.166338 5884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:06.166455 5887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:06.168390 5876 server_base.cc:1034] running on GCE node
I20260810 14:39:06.169199 5876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:06.170586 5876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:06.171859 5876 hybrid_clock.cc:648] HybridClock initialized: now 1786372746171794 us; error 73 us; skew 500 ppm
I20260810 14:39:06.172302 5876 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:39:06.175333 5876 webserver.cc:460] Webserver started at http://127.21.55.66:46015/ using document root <none> and password file <none>
I20260810 14:39:06.176214 5876 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:06.176366 5876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:06.176767 5876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:06.180433 5876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "6e64232c9f564a9b8dc1389b8a92f995"
format_stamp: "Formatted at 2026-08-10 14:39:06 on dist-test-slave-h6n0"
server_key: "dd3b4a1eac80db856b287f2e27c7e3fc"
server_key_iv: "4b5c45286c831d81b779f79249335648"
server_key_version: "encryptionkey@0"
I20260810 14:39:06.181407 5876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "6e64232c9f564a9b8dc1389b8a92f995"
format_stamp: "Formatted at 2026-08-10 14:39:06 on dist-test-slave-h6n0"
server_key: "dd3b4a1eac80db856b287f2e27c7e3fc"
server_key_iv: "4b5c45286c831d81b779f79249335648"
server_key_version: "encryptionkey@0"
I20260810 14:39:06.188431 5876 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260810 14:39:06.193248 5899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:06.195233 5876 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 14:39:06.195472 5876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "6e64232c9f564a9b8dc1389b8a92f995"
format_stamp: "Formatted at 2026-08-10 14:39:06 on dist-test-slave-h6n0"
server_key: "dd3b4a1eac80db856b287f2e27c7e3fc"
server_key_iv: "4b5c45286c831d81b779f79249335648"
server_key_version: "encryptionkey@0"
I20260810 14:39:06.195765 5876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:06.202878 5876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:06.203873 5876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:06.204221 5876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:06.205384 5876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:39:06.207166 5876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:39:06.207310 5876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:06.207463 5876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:39:06.207554 5876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:06.250401 5876 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:41915
I20260810 14:39:06.250460 6108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:41915 every 8 connection(s)
I20260810 14:39:06.252206 5876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:39:06.261795 5873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:06.255418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:53263 (local address 127.21.55.126:45539)
0810 14:39:06.255699 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:06.255705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:06.257277 (+ 1572us) server_negotiation.cc:407] Connection header received
0810 14:39:06.259274 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:06.259279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:06.259341 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:06.259487 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:06.261067 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:06.261071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:06.261074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:06.261090 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:06.261124 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:06.261136 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:06.261254 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:06.261515 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 14:39:06.261579 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260810 14:39:06.261861 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 5876
I20260810 14:39:06.262091 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:39:06.262771 21725 external_mini_cluster.cc:1428] Setting key f711603486aaf1af410255040dedc9d6
I20260810 14:39:06.263454 6115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:06.255910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45539 (local address 127.21.55.66:53263)
0810 14:39:06.257070 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:06.257118 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:06.258968 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:06.259732 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:06.259741 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:06.259844 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:06.260724 (+ 880us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:39:06.260727 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:06.260772 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:39:06.260878 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:39:06.261258 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:39:06.261267 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:39:06.261273 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:39:06.261631 (+ 358us) client_negotiation.cc:240] Negotiation successful
0810 14:39:06.261957 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":358,"threads_started":1}
I20260810 14:39:06.265128 6109 heartbeater.cc:347] Connected to a master server at 127.21.55.126:45539
I20260810 14:39:06.265532 6109 heartbeater.cc:464] Registering TS with master...
I20260810 14:39:06.266397 6109 heartbeater.cc:511] Master 127.21.55.126:45539 requested a full tablet report, sending...
I20260810 14:39:06.266736 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45539
--builtin_ntp_servers=127.21.55.84:43125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 14:39:06.268376 5537 ts_manager.cc:194] Registered new tserver with Master: 6e64232c9f564a9b8dc1389b8a92f995 (127.21.55.66:41915)
W20260810 14:39:06.626338 6116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:06.626649 6116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:06.626796 6116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:06.634487 6116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:06.634625 6116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:06.634804 6116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:39:06.646713 6116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:45539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:06.648159 6116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:06.650043 6116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:06.665266 6124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:06.665355 6127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:06.665355 6125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:06.666261 6116 server_base.cc:1034] running on GCE node
I20260810 14:39:06.667062 6116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:06.668128 6116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:06.669380 6116 hybrid_clock.cc:648] HybridClock initialized: now 1786372746669323 us; error 56 us; skew 500 ppm
I20260810 14:39:06.669724 6116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:39:06.672310 6116 webserver.cc:460] Webserver started at http://127.21.55.67:36103/ using document root <none> and password file <none>
I20260810 14:39:06.673056 6116 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:06.673198 6116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:06.673539 6116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:06.676522 6116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "da94fbe5d9934dfaa918fae683a1cad3"
format_stamp: "Formatted at 2026-08-10 14:39:06 on dist-test-slave-h6n0"
server_key: "da7fec2f7e1a1b44054e698ef0dcc14d"
server_key_iv: "20af274ffb3e049cae6fde89dec844fd"
server_key_version: "encryptionkey@0"
I20260810 14:39:06.677352 6116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "da94fbe5d9934dfaa918fae683a1cad3"
format_stamp: "Formatted at 2026-08-10 14:39:06 on dist-test-slave-h6n0"
server_key: "da7fec2f7e1a1b44054e698ef0dcc14d"
server_key_iv: "20af274ffb3e049cae6fde89dec844fd"
server_key_version: "encryptionkey@0"
I20260810 14:39:06.683115 6116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260810 14:39:06.687781 6138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:06.689937 6116 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:39:06.690167 6116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "da94fbe5d9934dfaa918fae683a1cad3"
format_stamp: "Formatted at 2026-08-10 14:39:06 on dist-test-slave-h6n0"
server_key: "da7fec2f7e1a1b44054e698ef0dcc14d"
server_key_iv: "20af274ffb3e049cae6fde89dec844fd"
server_key_version: "encryptionkey@0"
I20260810 14:39:06.690415 6116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:06.696825 6116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:06.697642 6116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:06.697932 6116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:06.699011 6116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:39:06.700914 6116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:39:06.701037 6116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:06.701174 6116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:39:06.701277 6116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:06.741897 6116 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:40727
I20260810 14:39:06.742049 6330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:40727 every 8 connection(s)
I20260810 14:39:06.743455 6116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:39:06.753435 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 6116
I20260810 14:39:06.753703 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:39:06.754501 21725 external_mini_cluster.cc:1428] Setting key f055c6055430316e2f6443a4daf6eb67
I20260810 14:39:06.755162 5873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:06.747146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:41447 (local address 127.21.55.126:45539)
0810 14:39:06.747341 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:06.747346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:06.749248 (+ 1902us) server_negotiation.cc:407] Connection header received
0810 14:39:06.751823 (+ 2575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:06.751832 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:06.751957 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:06.752142 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:06.754103 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:06.754114 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:06.754120 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:06.754148 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:06.754216 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:06.754234 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:06.754366 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:06.754851 (+ 485us) server_negotiation.cc:299] Negotiation successful
0810 14:39:06.754914 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 14:39:06.756925 6333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:06.747820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:45539 (local address 127.21.55.67:41447)
0810 14:39:06.749047 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:06.749094 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:06.751380 (+ 2286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:06.752475 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:06.752488 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:06.752656 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:06.753617 (+ 961us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:39:06.753620 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:06.753661 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:39:06.753808 (+ 147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:39:06.754373 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:39:06.754392 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:39:06.754403 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 14:39:06.754962 (+ 559us) client_negotiation.cc:240] Negotiation successful
0810 14:39:06.755352 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":414,"threads_started":1}
I20260810 14:39:06.758755 6331 heartbeater.cc:347] Connected to a master server at 127.21.55.126:45539
I20260810 14:39:06.759164 6331 heartbeater.cc:464] Registering TS with master...
I20260810 14:39:06.759979 6331 heartbeater.cc:511] Master 127.21.55.126:45539 requested a full tablet report, sending...
I20260810 14:39:06.761600 5535 ts_manager.cc:194] Registered new tserver with Master: da94fbe5d9934dfaa918fae683a1cad3 (127.21.55.67:40727)
I20260810 14:39:06.770943 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 14:39:06.778080 5873 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:39:06.776244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59882 (local address 127.21.55.126:45539)
0810 14:39:06.776407 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:06.776411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:06.777040 (+ 629us) server_negotiation.cc:407] Connection header received
0810 14:39:06.777257 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:06.777263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:06.777330 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:06.777437 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:06.777806 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59882: BlockingRecv error: recv got EOF from 127.0.0.1:59882 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 14:39:06.778832 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 5635
I20260810 14:39:06.784452 6342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:06.779935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50640 (local address 127.21.55.65:39761)
0810 14:39:06.780708 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:06.780717 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:06.780759 (+ 42us) server_negotiation.cc:407] Connection header received
0810 14:39:06.780862 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:06.780870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:06.781169 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:06.781345 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:06.783009 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:06.783018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:06.783034 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:06.783052 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:06.783173 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:06.783198 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:06.783422 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:06.783854 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 14:39:06.784017 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":346,"threads_started":1}
I20260810 14:39:06.786147 5859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:06.788748 5867 heartbeater.cc:503] Master 127.21.55.126:45539 was elected leader, sending a full tablet report...
I20260810 14:39:06.883404 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 5635
I20260810 14:39:06.916360 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 5876
I20260810 14:39:06.920591 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:06.917565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44664 (local address 127.21.55.66:41915)
0810 14:39:06.918178 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:06.918186 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:06.918214 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:39:06.918284 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:06.918290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:06.918523 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:06.918774 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:06.919835 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:06.919841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:06.919853 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:06.919872 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:06.919915 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:06.919930 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:06.920088 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:06.920200 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 14:39:06.920309 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":290,"threads_started":1}
I20260810 14:39:06.922439 6099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:07.017946 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 5876
I20260810 14:39:07.047349 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 6116
I20260810 14:39:07.052201 6346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:07.048537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48622 (local address 127.21.55.67:40727)
0810 14:39:07.049295 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:07.049304 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:07.049343 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:39:07.049422 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:07.049430 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:07.049723 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:07.049936 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:07.051313 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:07.051321 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:07.051334 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:39:07.051347 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:07.051397 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:39:07.051413 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:07.051639 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:07.051716 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 14:39:07.051849 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":333,"threads_started":1}
I20260810 14:39:07.053843 6323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:07.146430 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 6116
I20260810 14:39:07.172346 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 5477
I20260810 14:39:07.173676 5584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:07.276901 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 5477
2026-08-10T14:39:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2727 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:39:07 dist-test-slave-h6n0 krb5kdc[6352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:39:07 dist-test-slave-h6n0 krb5kdc[6352](info): set up 2 sockets
Aug 10 14:39:07 dist-test-slave-h6n0 krb5kdc[6352](info): commencing operation
krb5kdc: starting...
W20260810 14:39:08.687294 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.358s user 0.002s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:39:08 dist-test-slave-h6n0 krb5kdc[6352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:39:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:39:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:39:09.027328 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:33637
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:33637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:39:09.402599 6375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:09.402990 6375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:09.403193 6375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:09.411370 6375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:39:09.411525 6375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:09.411674 6375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:09.411818 6375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:39:09.411912 6375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:39:09.423023 6375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:33637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:33637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:09.424803 6375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:09.426416 6375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:09.441241 6386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:09.441219 6387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:09.441206 6391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:09.447409 6375 server_base.cc:1034] running on GCE node
I20260810 14:39:09.448561 6375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:09.450443 6375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:09.451730 6375 hybrid_clock.cc:648] HybridClock initialized: now 1786372749451676 us; error 38 us; skew 500 ppm
I20260810 14:39:09.452159 6375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:39:09 dist-test-slave-h6n0 krb5kdc[6352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372749, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:39:09.459713 6375 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:39:09.461762 6375 webserver.cc:460] Webserver started at http://127.21.55.126:42151/ using document root <none> and password file <none>
I20260810 14:39:09.462767 6375 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:09.462936 6375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:09.463315 6375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:09.466939 6375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "de5d51540cc84e39a6b51fd2d1ca8884"
format_stamp: "Formatted at 2026-08-10 14:39:09 on dist-test-slave-h6n0"
server_key: "83b5c05c5325be3ec7044162fe4a47bd"
server_key_iv: "c24028f74cbd84a547e8fb0e0a531208"
server_key_version: "encryptionkey@0"
I20260810 14:39:09.467880 6375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "de5d51540cc84e39a6b51fd2d1ca8884"
format_stamp: "Formatted at 2026-08-10 14:39:09 on dist-test-slave-h6n0"
server_key: "83b5c05c5325be3ec7044162fe4a47bd"
server_key_iv: "c24028f74cbd84a547e8fb0e0a531208"
server_key_version: "encryptionkey@0"
I20260810 14:39:09.474016 6375 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260810 14:39:09.478248 6410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:09.480279 6375 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:39:09.480533 6375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "de5d51540cc84e39a6b51fd2d1ca8884"
format_stamp: "Formatted at 2026-08-10 14:39:09 on dist-test-slave-h6n0"
server_key: "83b5c05c5325be3ec7044162fe4a47bd"
server_key_iv: "c24028f74cbd84a547e8fb0e0a531208"
server_key_version: "encryptionkey@0"
I20260810 14:39:09.480796 6375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:09.523358 6375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:09.529465 6375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:09.529857 6375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:09.554281 6375 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:33637
I20260810 14:39:09.554338 6499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:33637 every 8 connection(s)
I20260810 14:39:09.556113 6375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:39:09.560397 6500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:39:09.564422 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 6375
I20260810 14:39:09.564698 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:39:09.565580 21725 external_mini_cluster.cc:1428] Setting key a99fea76790f9414ed2e6b48d4606d97
I20260810 14:39:09.573524 6500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884: Bootstrap starting.
I20260810 14:39:09.577526 6500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884: Neither blocks nor log segments found. Creating new log.
I20260810 14:39:09.578873 6500 log.cc:824] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884: Log is configured to *not* fsync() on all Append() calls
I20260810 14:39:09.583058 6500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884: No bootstrap required, opened a new log
Aug 10 14:39:09 dist-test-slave-h6n0 krb5kdc[6352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:39:09.593437 6500 raft_consensus.cc:348] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5d51540cc84e39a6b51fd2d1ca8884" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33637 } }
I20260810 14:39:09.594020 6500 raft_consensus.cc:374] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:39:09.594147 6500 raft_consensus.cc:729] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de5d51540cc84e39a6b51fd2d1ca8884, State: Initialized, Role: FOLLOWER
I20260810 14:39:09.595208 6500 consensus_queue.cc:260] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5d51540cc84e39a6b51fd2d1ca8884" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33637 } }
I20260810 14:39:09.595491 6500 raft_consensus.cc:388] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:39:09.595698 6500 raft_consensus.cc:482] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:39:09.595916 6500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:39:09.596777 6506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:09.568329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.21.55.126:33637)
0810 14:39:09.569361 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:09.569376 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:09.569430 (+ 54us) server_negotiation.cc:407] Connection header received
0810 14:39:09.571970 (+ 2540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:09.572002 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:09.572505 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:09.572935 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:09.574915 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:09.576430 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:09.578489 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:09.579577 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:09.579828 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:09.585514 (+ 5686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:09.585528 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:09.585551 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:39:09.585614 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:09.590920 (+ 5306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:09.591887 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:09.591895 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:09.591908 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:09.592155 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:09.592621 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:09.592625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:09.592627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:09.593382 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:09.593676 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:09.594343 (+ 667us) server_negotiation.cc:299] Negotiation successful
0810 14:39:09.594955 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":503,"threads_started":1}
I20260810 14:39:09.598614 6500 raft_consensus.cc:504] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5d51540cc84e39a6b51fd2d1ca8884" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33637 } }
I20260810 14:39:09.599215 6500 leader_election.cc:302] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de5d51540cc84e39a6b51fd2d1ca8884; no voters:
I20260810 14:39:09.599958 6500 leader_election.cc:288] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:39:09.600181 6508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:39:09.601981 6508 raft_consensus.cc:686] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [term 1 LEADER]: Becoming Leader. State: Replica: de5d51540cc84e39a6b51fd2d1ca8884, State: Running, Role: LEADER
I20260810 14:39:09.602669 6500 sys_catalog.cc:564] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:39:09.602809 6508 consensus_queue.cc:237] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5d51540cc84e39a6b51fd2d1ca8884" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33637 } }
I20260810 14:39:09.606761 6509 sys_catalog.cc:455] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de5d51540cc84e39a6b51fd2d1ca8884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5d51540cc84e39a6b51fd2d1ca8884" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33637 } } }
I20260810 14:39:09.607250 6509 sys_catalog.cc:458] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [sys.catalog]: This master's current role is: LEADER
I20260810 14:39:09.610741 6511 sys_catalog.cc:455] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de5d51540cc84e39a6b51fd2d1ca8884. Latest consensus state: current_term: 1 leader_uuid: "de5d51540cc84e39a6b51fd2d1ca8884" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de5d51540cc84e39a6b51fd2d1ca8884" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33637 } } }
I20260810 14:39:09.611207 6511 sys_catalog.cc:458] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884 [sys.catalog]: This master's current role is: LEADER
I20260810 14:39:09.613235 6519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:39:09.618506 6519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:39:09.630327 6519 catalog_manager.cc:1347] Generated new cluster ID: c26e1738690647119cd38cf0b130c6f8
I20260810 14:39:09.630460 6519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:39:09.665449 6519 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:39:09.667295 6519 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:39:09.675210 6519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884: Generated new TSK 0
I20260810 14:39:09.676394 6519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:39:09.760294 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33637
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:39:10.176234 6542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:10.176538 6542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:10.176646 6542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:10.184653 6542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:10.184927 6542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:10.185139 6542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:39:10.195400 6542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:10.196768 6542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:10.198187 6542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:10.212570 6547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:10.212870 6548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:10.213400 6550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:10.213773 6542 server_base.cc:1034] running on GCE node
I20260810 14:39:10.214495 6542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:10.215653 6542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:10.216926 6542 hybrid_clock.cc:648] HybridClock initialized: now 1786372750216891 us; error 45 us; skew 500 ppm
I20260810 14:39:10.217294 6542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:39:10 dist-test-slave-h6n0 krb5kdc[6352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372750, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:39:10.221199 6542 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:39:10.223374 6542 webserver.cc:460] Webserver started at http://127.21.55.65:42041/ using document root <none> and password file <none>
I20260810 14:39:10.224200 6542 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:10.224509 6542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:10.225189 6542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:10.229343 6542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "07db33eeb2034681bdd94bdeb2190926"
format_stamp: "Formatted at 2026-08-10 14:39:10 on dist-test-slave-h6n0"
server_key: "6d18976b7d59ddd3221910efc5233c0d"
server_key_iv: "26ddf3d6bbe6c763bad79f224f1cff73"
server_key_version: "encryptionkey@0"
I20260810 14:39:10.230406 6542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "07db33eeb2034681bdd94bdeb2190926"
format_stamp: "Formatted at 2026-08-10 14:39:10 on dist-test-slave-h6n0"
server_key: "6d18976b7d59ddd3221910efc5233c0d"
server_key_iv: "26ddf3d6bbe6c763bad79f224f1cff73"
server_key_version: "encryptionkey@0"
I20260810 14:39:10.237617 6542 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260810 14:39:10.243088 6562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:10.245057 6542 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:39:10.245280 6542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "07db33eeb2034681bdd94bdeb2190926"
format_stamp: "Formatted at 2026-08-10 14:39:10 on dist-test-slave-h6n0"
server_key: "6d18976b7d59ddd3221910efc5233c0d"
server_key_iv: "26ddf3d6bbe6c763bad79f224f1cff73"
server_key_version: "encryptionkey@0"
I20260810 14:39:10.245534 6542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:10.306354 6542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:10.312050 6542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:10.312463 6542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:10.313704 6542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:39:10.315400 6542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:39:10.315526 6542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:10.315702 6542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:39:10.315793 6542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:10.357262 6542 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:33733
I20260810 14:39:10.357367 6743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:33733 every 8 connection(s)
I20260810 14:39:10.358697 6542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:39:10.368938 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 6542
I20260810 14:39:10.369220 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:39:10.370144 21725 external_mini_cluster.cc:1428] Setting key 4732bd415773f7f908333ac5ef091627
Aug 10 14:39:10 dist-test-slave-h6n0 krb5kdc[6352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372750, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:39:10.382193 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:10.361358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:40981 (local address 127.21.55.126:33637)
0810 14:39:10.361904 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:10.361909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:10.363269 (+ 1360us) server_negotiation.cc:407] Connection header received
0810 14:39:10.365167 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:10.365171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:10.365243 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:10.365345 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:10.367069 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:10.367841 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:10.369554 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:10.370240 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:10.370381 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:10.375663 (+ 5282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:10.375669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:10.375673 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:39:10.375716 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:10.378831 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:10.379731 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:10.379743 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:10.379746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:10.379871 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:10.380361 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:10.380366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:10.380368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:10.380633 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:10.380799 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:10.381549 (+ 750us) server_negotiation.cc:299] Negotiation successful
0810 14:39:10.381783 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":296,"threads_started":1}
I20260810 14:39:10.383287 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:10.361965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33637 (local address 127.21.55.65:40981)
0810 14:39:10.363066 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:10.363144 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:10.364885 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:10.365617 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:10.365625 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:10.366182 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:10.366868 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:10.366886 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:10.367994 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:10.367998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:10.369387 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:10.369400 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:10.370501 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:10.370506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:10.370787 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:10.371690 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:39:10.371728 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:10.375310 (+ 3582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:39:10.379017 (+ 3707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:39:10.379030 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:39:10.379051 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:39:10.379584 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:39:10.379994 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:39:10.379999 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:39:10.380003 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:39:10.380220 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:39:10.380779 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:39:10.380787 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:39:10.381143 (+ 356us) client_negotiation.cc:769] Sending connection context
0810 14:39:10.381427 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 14:39:10.382025 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":379,"threads_started":1}
I20260810 14:39:10.385116 6745 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33637
I20260810 14:39:10.385457 6745 heartbeater.cc:464] Registering TS with master...
I20260810 14:39:10.386364 6745 heartbeater.cc:511] Master 127.21.55.126:33637 requested a full tablet report, sending...
I20260810 14:39:10.389050 6436 ts_manager.cc:194] Registered new tserver with Master: 07db33eeb2034681bdd94bdeb2190926 (127.21.55.65:33733)
I20260810 14:39:10.391309 6436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:40981
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:39:10.439154 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33637
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:39:10.831666 6754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:10.831947 6754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:10.832104 6754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:10.839285 6754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:10.839401 6754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:10.839600 6754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:39:10.849422 6754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:10.850785 6754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:10.852070 6754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:10.866223 6768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:10.866324 6766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:10.866452 6764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:10.867682 6754 server_base.cc:1034] running on GCE node
I20260810 14:39:10.868624 6754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:10.869796 6754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:10.871124 6754 hybrid_clock.cc:648] HybridClock initialized: now 1786372750871051 us; error 79 us; skew 500 ppm
I20260810 14:39:10.871462 6754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:39:10 dist-test-slave-h6n0 krb5kdc[6352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372750, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:39:10.875725 6754 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:39:10.877419 6754 webserver.cc:460] Webserver started at http://127.21.55.66:35535/ using document root <none> and password file <none>
I20260810 14:39:10.878163 6754 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:10.878312 6754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:10.878700 6754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:10.881743 6754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "89908f912516448c8f6fc0c1a7a4d165"
format_stamp: "Formatted at 2026-08-10 14:39:10 on dist-test-slave-h6n0"
server_key: "d98b9af812f7db3cfefff904fc79603c"
server_key_iv: "bb174fec8a6aaad5da8cde84e53a5b62"
server_key_version: "encryptionkey@0"
I20260810 14:39:10.882599 6754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "89908f912516448c8f6fc0c1a7a4d165"
format_stamp: "Formatted at 2026-08-10 14:39:10 on dist-test-slave-h6n0"
server_key: "d98b9af812f7db3cfefff904fc79603c"
server_key_iv: "bb174fec8a6aaad5da8cde84e53a5b62"
server_key_version: "encryptionkey@0"
I20260810 14:39:10.887948 6754 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 14:39:10.891472 6781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:10.892843 6754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:39:10.892988 6754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "89908f912516448c8f6fc0c1a7a4d165"
format_stamp: "Formatted at 2026-08-10 14:39:10 on dist-test-slave-h6n0"
server_key: "d98b9af812f7db3cfefff904fc79603c"
server_key_iv: "bb174fec8a6aaad5da8cde84e53a5b62"
server_key_version: "encryptionkey@0"
I20260810 14:39:10.893198 6754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:10.919997 6754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:10.925547 6754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:10.925830 6754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:10.926851 6754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:39:10.928309 6754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:39:10.928388 6754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:10.928474 6754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:39:10.928519 6754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:10.966082 6754 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:39405
I20260810 14:39:10.966280 6971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:39405 every 8 connection(s)
I20260810 14:39:10.967504 6754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:39:10.970993 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 6754
I20260810 14:39:10.971231 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:39:10.971693 21725 external_mini_cluster.cc:1428] Setting key f3a1b0d238ddf116d4d5d32ed6534a16
Aug 10 14:39:10 dist-test-slave-h6n0 krb5kdc[6352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372750, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:39:10.988976 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:10.970085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:35401 (local address 127.21.55.126:33637)
0810 14:39:10.970590 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:10.970595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:10.971730 (+ 1135us) server_negotiation.cc:407] Connection header received
0810 14:39:10.973828 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:10.973834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:10.973924 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:10.974036 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:10.975629 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:10.976783 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:10.978369 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:10.979114 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:10.979252 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:10.982793 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:10.982800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:10.982804 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:39:10.982843 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:10.985677 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:10.986600 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:10.986604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:10.986607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:10.986728 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:10.987196 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:10.987200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:10.987202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:10.987472 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:10.987641 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:10.988353 (+ 712us) server_negotiation.cc:299] Negotiation successful
0810 14:39:10.988606 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":235,"threads_started":1}
I20260810 14:39:10.989946 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:10.970710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33637 (local address 127.21.55.66:35401)
0810 14:39:10.971546 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:10.971600 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:10.973526 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:10.974254 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:10.974262 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:10.974847 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:10.975424 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:10.975441 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:10.976966 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:10.976970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:10.978207 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:10.978220 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:10.979344 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:10.979349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:10.979543 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:10.980272 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:39:10.980293 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:10.982501 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:39:10.985809 (+ 3308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:39:10.985818 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:39:10.985833 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:39:10.986420 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:39:10.986851 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:39:10.986856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:39:10.986860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:39:10.987065 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:39:10.987622 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:39:10.987630 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:39:10.987964 (+ 334us) client_negotiation.cc:769] Sending connection context
0810 14:39:10.988254 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 14:39:10.988790 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":359,"threads_started":1}
I20260810 14:39:10.991501 6975 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33637
I20260810 14:39:10.991838 6975 heartbeater.cc:464] Registering TS with master...
I20260810 14:39:10.992769 6975 heartbeater.cc:511] Master 127.21.55.126:33637 requested a full tablet report, sending...
I20260810 14:39:10.994393 6436 ts_manager.cc:194] Registered new tserver with Master: 89908f912516448c8f6fc0c1a7a4d165 (127.21.55.66:39405)
I20260810 14:39:10.995729 6436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:35401
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:39:11.039290 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33637
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:43527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:39:11.395490 6745 heartbeater.cc:503] Master 127.21.55.126:33637 was elected leader, sending a full tablet report...
W20260810 14:39:11.398113 6982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:39:11.398382 6982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:39:11.398624 6982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:39:11.406320 6982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:39:11.406471 6982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:39:11.406705 6982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:39:11.416723 6982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:39:11.418187 6982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:39:11.419516 6982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:39:11.432718 6992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:11.432766 6994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:39:11.432720 6991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:39:11.434078 6982 server_base.cc:1034] running on GCE node
I20260810 14:39:11.434798 6982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:39:11.435753 6982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:39:11.437016 6982 hybrid_clock.cc:648] HybridClock initialized: now 1786372751436930 us; error 55 us; skew 500 ppm
I20260810 14:39:11.437336 6982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:39:11 dist-test-slave-h6n0 krb5kdc[6352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372751, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:39:11.440979 6982 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:39:11.442619 6982 webserver.cc:460] Webserver started at http://127.21.55.67:32943/ using document root <none> and password file <none>
I20260810 14:39:11.443356 6982 fs_manager.cc:359] Metadata directory not provided
I20260810 14:39:11.443532 6982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:39:11.443876 6982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:39:11.447351 6982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "395a500db03846d98ecf72fbfe98bbbe"
format_stamp: "Formatted at 2026-08-10 14:39:11 on dist-test-slave-h6n0"
server_key: "05e23c8561094bf78a6b9c7b815e33a7"
server_key_iv: "440b9648868546821ef602eb3bc8223c"
server_key_version: "encryptionkey@0"
I20260810 14:39:11.448201 6982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "395a500db03846d98ecf72fbfe98bbbe"
format_stamp: "Formatted at 2026-08-10 14:39:11 on dist-test-slave-h6n0"
server_key: "05e23c8561094bf78a6b9c7b815e33a7"
server_key_iv: "440b9648868546821ef602eb3bc8223c"
server_key_version: "encryptionkey@0"
I20260810 14:39:11.453786 6982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260810 14:39:11.457320 7005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:11.458774 6982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:39:11.458963 6982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "395a500db03846d98ecf72fbfe98bbbe"
format_stamp: "Formatted at 2026-08-10 14:39:11 on dist-test-slave-h6n0"
server_key: "05e23c8561094bf78a6b9c7b815e33a7"
server_key_iv: "440b9648868546821ef602eb3bc8223c"
server_key_version: "encryptionkey@0"
I20260810 14:39:11.459156 6982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:39:11.478368 6982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:39:11.483515 6982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:39:11.484030 6982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:39:11.485359 6982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:39:11.486899 6982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:39:11.487016 6982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:11.487174 6982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:39:11.487246 6982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:39:11.526669 6982 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:44129
I20260810 14:39:11.526693 7188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:44129 every 8 connection(s)
I20260810 14:39:11.528119 6982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:39:11.530277 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 6982
I20260810 14:39:11.530525 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:39:11.531266 21725 external_mini_cluster.cc:1428] Setting key 2fc816af4b2361dda041b651ab74198d
Aug 10 14:39:11 dist-test-slave-h6n0 krb5kdc[6352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372751, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:39:11.550304 7194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.531421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:59111 (local address 127.21.55.126:33637)
0810 14:39:11.531961 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:11.531969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:11.533200 (+ 1231us) server_negotiation.cc:407] Connection header received
0810 14:39:11.535559 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:11.535565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:11.535649 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:11.535763 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:11.537302 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.538133 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.539754 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.540168 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.540263 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.543849 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:11.543856 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:11.543859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:39:11.543914 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:11.547361 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:11.548123 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:11.548128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:11.548130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:11.548251 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:11.548628 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:11.548631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:11.548633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:11.548893 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:11.549010 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:11.549874 (+ 864us) server_negotiation.cc:299] Negotiation successful
0810 14:39:11.550034 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":268,"threads_started":1}
I20260810 14:39:11.551340 7195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.532059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33637 (local address 127.21.55.67:59111)
0810 14:39:11.533036 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:11.533094 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:11.535297 (+ 2203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:11.536006 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:11.536014 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:11.536518 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:39:11.537107 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.537131 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.538301 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.538305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.539616 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.539629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.540370 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.540375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.540560 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.541234 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:39:11.541261 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:39:11.543462 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:39:11.547507 (+ 4045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:39:11.547514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:39:11.547525 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:39:11.547979 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:39:11.548341 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:39:11.548344 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:39:11.548347 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:39:11.548524 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:39:11.549032 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:39:11.549039 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:39:11.549424 (+ 385us) client_negotiation.cc:769] Sending connection context
0810 14:39:11.549725 (+ 301us) client_negotiation.cc:240] Negotiation successful
0810 14:39:11.550163 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":386,"threads_started":1}
I20260810 14:39:11.553462 7189 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33637
I20260810 14:39:11.554050 7189 heartbeater.cc:464] Registering TS with master...
I20260810 14:39:11.555255 7189 heartbeater.cc:511] Master 127.21.55.126:33637 requested a full tablet report, sending...
I20260810 14:39:11.556892 6436 ts_manager.cc:194] Registered new tserver with Master: 395a500db03846d98ecf72fbfe98bbbe (127.21.55.67:44129)
I20260810 14:39:11.557816 6436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:59111
I20260810 14:39:11.559808 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:39:11.577833 7194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.564598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48478 (local address 127.21.55.126:33637)
0810 14:39:11.564783 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:11.564794 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:11.565396 (+ 602us) server_negotiation.cc:407] Connection header received
0810 14:39:11.565491 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:11.565494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:11.565565 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:11.565651 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:11.567115 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.567839 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.569166 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.569674 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.569834 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.572129 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:11.572135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:11.572139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:39:11.572176 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:11.575441 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:11.576128 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:11.576132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:11.576134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:11.576258 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:11.576596 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:11.576599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:11.576601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:11.576906 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:11.577089 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:11.577427 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 14:39:11.577617 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:39:11.582120 6435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:39:11.589948 6435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:39:11.625243 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.603342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46758 (local address 127.21.55.67:44129)
0810 14:39:11.614450 (+ 11108us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:11.614465 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:11.615459 (+ 994us) server_negotiation.cc:407] Connection header received
0810 14:39:11.615888 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:11.615895 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:11.616166 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:11.616455 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:39:11.617175 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.619432 (+ 2257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.622353 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.623886 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.624017 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.624258 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:11.624649 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 14:39:11.624839 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10841,"thread_start_us":11168,"threads_started":1}
I20260810 14:39:11.625268 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.602988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:44129 (local address 127.0.0.1:46758)
0810 14:39:11.615292 (+ 12304us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:11.615333 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:11.615628 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:11.616500 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:11.616508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:11.616540 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:39:11.616976 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.616987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.619566 (+ 2579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.619572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.622172 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.622186 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.624109 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.624119 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.624357 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.624431 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 14:39:11.624587 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 14:39:11.624798 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12069,"mutex_wait_us":6,"thread_start_us":11903,"threads_started":1}
I20260810 14:39:11.627051 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.603614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:39405 (local address 127.0.0.1:44656)
0810 14:39:11.618630 (+ 15016us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:11.618648 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:11.618768 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:11.619542 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:11.619546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:11.619567 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:39:11.620451 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.620468 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.621943 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.621949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.624039 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.624054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.626073 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.626080 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.626268 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.626326 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 14:39:11.626423 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 14:39:11.626600 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14880,"mutex_wait_us":24,"thread_start_us":14125,"threads_started":1}
I20260810 14:39:11.627916 7215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.604137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44656 (local address 127.21.55.66:39405)
0810 14:39:11.617563 (+ 13426us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:11.617571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:11.618759 (+ 1188us) server_negotiation.cc:407] Connection header received
0810 14:39:11.618899 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:11.618905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:11.619117 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:11.619300 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:39:11.620692 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.621778 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.624259 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.625853 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.626015 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.626240 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:11.627332 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0810 14:39:11.627528 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13260,"thread_start_us":13163,"threads_started":1}
I20260810 14:39:11.628701 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.603348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:33733 (local address 127.0.0.1:60912)
0810 14:39:11.618024 (+ 14676us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:11.618043 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:11.618144 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:11.619510 (+ 1366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:11.619517 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:11.619546 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:39:11.620468 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.620479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.622344 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.622349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.624503 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.624515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.627839 (+ 3324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.627848 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.628084 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.628117 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:39:11.628209 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 14:39:11.628332 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14525,"mutex_wait_us":29,"thread_start_us":15047,"threads_started":1}
I20260810 14:39:11.629047 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.604127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.21.55.65:33733)
0810 14:39:11.618572 (+ 14445us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:11.618581 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:11.618614 (+ 33us) server_negotiation.cc:407] Connection header received
0810 14:39:11.618746 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:11.618753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:11.618991 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:11.619227 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:39:11.620685 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.622174 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.624700 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.627279 (+ 2579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.627437 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.627685 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:11.628311 (+ 626us) server_negotiation.cc:299] Negotiation successful
0810 14:39:11.628494 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14243,"thread_start_us":14052,"threads_started":1}
I20260810 14:39:11.631484 7081 tablet_service.cc:1467] Processing CreateTablet for tablet f14f519e7c994adfbe67a68bdad6f889 (DEFAULT_TABLE table=test-table [id=ab819ea42e314eb7bfe6993a5ecf7245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:39:11.631453 6868 tablet_service.cc:1467] Processing CreateTablet for tablet f14f519e7c994adfbe67a68bdad6f889 (DEFAULT_TABLE table=test-table [id=ab819ea42e314eb7bfe6993a5ecf7245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:39:11.632975 6868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14f519e7c994adfbe67a68bdad6f889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:39:11.633000 7081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14f519e7c994adfbe67a68bdad6f889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:39:11.632714 6642 tablet_service.cc:1467] Processing CreateTablet for tablet f14f519e7c994adfbe67a68bdad6f889 (DEFAULT_TABLE table=test-table [id=ab819ea42e314eb7bfe6993a5ecf7245]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:39:11.634037 6642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14f519e7c994adfbe67a68bdad6f889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:39:11.646469 7220 tablet_bootstrap.cc:492] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: Bootstrap starting.
I20260810 14:39:11.647754 7222 tablet_bootstrap.cc:492] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: Bootstrap starting.
I20260810 14:39:11.649436 7225 tablet_bootstrap.cc:492] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: Bootstrap starting.
I20260810 14:39:11.650796 7220 tablet_bootstrap.cc:654] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: Neither blocks nor log segments found. Creating new log.
I20260810 14:39:11.651326 7222 tablet_bootstrap.cc:654] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: Neither blocks nor log segments found. Creating new log.
I20260810 14:39:11.652231 7225 tablet_bootstrap.cc:654] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: Neither blocks nor log segments found. Creating new log.
I20260810 14:39:11.652237 7220 log.cc:824] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: Log is configured to *not* fsync() on all Append() calls
I20260810 14:39:11.652472 7222 log.cc:824] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: Log is configured to *not* fsync() on all Append() calls
I20260810 14:39:11.653417 7225 log.cc:824] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: Log is configured to *not* fsync() on all Append() calls
I20260810 14:39:11.655552 7220 tablet_bootstrap.cc:492] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: No bootstrap required, opened a new log
I20260810 14:39:11.655841 7220 ts_tablet_manager.cc:1397] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 14:39:11.656141 7222 tablet_bootstrap.cc:492] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: No bootstrap required, opened a new log
I20260810 14:39:11.656239 7225 tablet_bootstrap.cc:492] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: No bootstrap required, opened a new log
I20260810 14:39:11.656402 7222 ts_tablet_manager.cc:1397] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 14:39:11.656498 7225 ts_tablet_manager.cc:1397] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 14:39:11.666720 7225 raft_consensus.cc:348] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395a500db03846d98ecf72fbfe98bbbe" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44129 } } peers { permanent_uuid: "07db33eeb2034681bdd94bdeb2190926" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33733 } } peers { permanent_uuid: "89908f912516448c8f6fc0c1a7a4d165" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39405 } }
I20260810 14:39:11.667304 7225 raft_consensus.cc:374] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:39:11.667433 7225 raft_consensus.cc:729] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07db33eeb2034681bdd94bdeb2190926, State: Initialized, Role: FOLLOWER
I20260810 14:39:11.667462 7220 raft_consensus.cc:348] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395a500db03846d98ecf72fbfe98bbbe" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44129 } } peers { permanent_uuid: "07db33eeb2034681bdd94bdeb2190926" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33733 } } peers { permanent_uuid: "89908f912516448c8f6fc0c1a7a4d165" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39405 } }
I20260810 14:39:11.668015 7220 raft_consensus.cc:374] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:39:11.668143 7220 raft_consensus.cc:729] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89908f912516448c8f6fc0c1a7a4d165, State: Initialized, Role: FOLLOWER
I20260810 14:39:11.667928 7222 raft_consensus.cc:348] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395a500db03846d98ecf72fbfe98bbbe" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44129 } } peers { permanent_uuid: "07db33eeb2034681bdd94bdeb2190926" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33733 } } peers { permanent_uuid: "89908f912516448c8f6fc0c1a7a4d165" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39405 } }
I20260810 14:39:11.668439 7222 raft_consensus.cc:374] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:39:11.668565 7222 raft_consensus.cc:729] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 395a500db03846d98ecf72fbfe98bbbe, State: Initialized, Role: FOLLOWER
I20260810 14:39:11.668383 7225 consensus_queue.cc:260] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395a500db03846d98ecf72fbfe98bbbe" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44129 } } peers { permanent_uuid: "07db33eeb2034681bdd94bdeb2190926" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33733 } } peers { permanent_uuid: "89908f912516448c8f6fc0c1a7a4d165" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39405 } }
I20260810 14:39:11.668998 7220 consensus_queue.cc:260] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395a500db03846d98ecf72fbfe98bbbe" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44129 } } peers { permanent_uuid: "07db33eeb2034681bdd94bdeb2190926" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33733 } } peers { permanent_uuid: "89908f912516448c8f6fc0c1a7a4d165" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39405 } }
I20260810 14:39:11.669368 7222 consensus_queue.cc:260] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395a500db03846d98ecf72fbfe98bbbe" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44129 } } peers { permanent_uuid: "07db33eeb2034681bdd94bdeb2190926" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33733 } } peers { permanent_uuid: "89908f912516448c8f6fc0c1a7a4d165" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39405 } }
I20260810 14:39:11.670504 7225 ts_tablet_manager.cc:1428] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260810 14:39:11.676096 6975 heartbeater.cc:503] Master 127.21.55.126:33637 was elected leader, sending a full tablet report...
I20260810 14:39:11.676463 7189 heartbeater.cc:503] Master 127.21.55.126:33637 was elected leader, sending a full tablet report...
I20260810 14:39:11.677336 7222 ts_tablet_manager.cc:1428] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: Time spent starting tablet: real 0.021s user 0.010s sys 0.012s
I20260810 14:39:11.678339 7220 ts_tablet_manager.cc:1428] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: Time spent starting tablet: real 0.022s user 0.010s sys 0.011s
W20260810 14:39:11.719759 6976 tablet.cc:2369] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:39:11.779966 7191 tablet.cc:2369] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:39:11.797721 7234 raft_consensus.cc:482] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:39:11.798087 7234 raft_consensus.cc:504] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395a500db03846d98ecf72fbfe98bbbe" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44129 } } peers { permanent_uuid: "07db33eeb2034681bdd94bdeb2190926" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33733 } } peers { permanent_uuid: "89908f912516448c8f6fc0c1a7a4d165" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39405 } }
I20260810 14:39:11.799988 7234 leader_election.cc:288] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 395a500db03846d98ecf72fbfe98bbbe (127.21.55.67:44129), 89908f912516448c8f6fc0c1a7a4d165 (127.21.55.66:39405)
I20260810 14:39:11.808668 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.800805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:44129 (local address 127.21.55.65:50363)
0810 14:39:11.801593 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:11.801627 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:11.801760 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:11.802123 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:11.802128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:11.802152 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:39:11.802608 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.802619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.803775 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.803781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.805386 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.805399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.807736 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.807742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.807868 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.807896 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 14:39:11.808045 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 14:39:11.808219 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":40,"thread_start_us":495,"threads_started":1}
I20260810 14:39:11.808591 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.800468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:50363 (local address 127.21.55.67:44129)
0810 14:39:11.800734 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:11.800741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:11.801730 (+ 989us) server_negotiation.cc:407] Connection header received
0810 14:39:11.801916 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:11.801920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:11.801981 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:11.802086 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:39:11.802783 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.803611 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.805719 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.807339 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.807569 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.807679 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:11.808100 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 14:39:11.808300 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260810 14:39:11.809075 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.800736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:39405 (local address 127.21.55.65:40711)
0810 14:39:11.801529 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0810 14:39:11.801559 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 14:39:11.801719 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:39:11.802087 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:39:11.802091 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:39:11.802116 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:39:11.802592 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.802610 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.803726 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.803730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.805927 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:39:11.805961 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.808191 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.808198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:39:11.808390 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.808425 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 14:39:11.808539 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 14:39:11.808687 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":103,"thread_start_us":407,"threads_started":1}
I20260810 14:39:11.809496 7215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.800465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:40711 (local address 127.21.55.66:39405)
0810 14:39:11.800697 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:11.800703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:11.801655 (+ 952us) server_negotiation.cc:407] Connection header received
0810 14:39:11.801873 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:11.801882 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:11.801957 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:11.802033 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:39:11.802745 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.803596 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.806350 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.807887 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.808092 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.808180 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:11.808596 (+ 416us) server_negotiation.cc:299] Negotiation successful
0810 14:39:11.809228 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:39:11.809969 7114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14f519e7c994adfbe67a68bdad6f889" candidate_uuid: "07db33eeb2034681bdd94bdeb2190926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "395a500db03846d98ecf72fbfe98bbbe" is_pre_election: true
I20260810 14:39:11.810318 6895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14f519e7c994adfbe67a68bdad6f889" candidate_uuid: "07db33eeb2034681bdd94bdeb2190926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89908f912516448c8f6fc0c1a7a4d165" is_pre_election: true
I20260810 14:39:11.810468 7114 raft_consensus.cc:2454] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07db33eeb2034681bdd94bdeb2190926 in term 0.
I20260810 14:39:11.810822 6895 raft_consensus.cc:2454] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07db33eeb2034681bdd94bdeb2190926 in term 0.
I20260810 14:39:11.811231 6564 leader_election.cc:302] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07db33eeb2034681bdd94bdeb2190926, 395a500db03846d98ecf72fbfe98bbbe; no voters:
I20260810 14:39:11.811641 7234 raft_consensus.cc:2781] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:39:11.811800 7234 raft_consensus.cc:482] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:39:11.811921 7234 raft_consensus.cc:3037] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:39:11.814155 7234 raft_consensus.cc:504] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395a500db03846d98ecf72fbfe98bbbe" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44129 } } peers { permanent_uuid: "07db33eeb2034681bdd94bdeb2190926" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33733 } } peers { permanent_uuid: "89908f912516448c8f6fc0c1a7a4d165" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39405 } }
I20260810 14:39:11.814813 7234 leader_election.cc:288] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [CANDIDATE]: Term 1 election: Requested vote from peers 395a500db03846d98ecf72fbfe98bbbe (127.21.55.67:44129), 89908f912516448c8f6fc0c1a7a4d165 (127.21.55.66:39405)
I20260810 14:39:11.815240 6895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14f519e7c994adfbe67a68bdad6f889" candidate_uuid: "07db33eeb2034681bdd94bdeb2190926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89908f912516448c8f6fc0c1a7a4d165"
I20260810 14:39:11.815271 7114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14f519e7c994adfbe67a68bdad6f889" candidate_uuid: "07db33eeb2034681bdd94bdeb2190926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "395a500db03846d98ecf72fbfe98bbbe"
I20260810 14:39:11.815434 6895 raft_consensus.cc:3037] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:39:11.815452 7114 raft_consensus.cc:3037] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:39:11.817606 7114 raft_consensus.cc:2454] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07db33eeb2034681bdd94bdeb2190926 in term 1.
I20260810 14:39:11.817611 6895 raft_consensus.cc:2454] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07db33eeb2034681bdd94bdeb2190926 in term 1.
I20260810 14:39:11.818002 6564 leader_election.cc:302] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07db33eeb2034681bdd94bdeb2190926, 395a500db03846d98ecf72fbfe98bbbe; no voters:
I20260810 14:39:11.818265 7234 raft_consensus.cc:2781] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:39:11.818799 7234 raft_consensus.cc:686] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 1 LEADER]: Becoming Leader. State: Replica: 07db33eeb2034681bdd94bdeb2190926, State: Running, Role: LEADER
I20260810 14:39:11.819278 7234 consensus_queue.cc:237] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395a500db03846d98ecf72fbfe98bbbe" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44129 } } peers { permanent_uuid: "07db33eeb2034681bdd94bdeb2190926" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33733 } } peers { permanent_uuid: "89908f912516448c8f6fc0c1a7a4d165" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39405 } }
I20260810 14:39:11.823846 6435 catalog_manager.cc:5495] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07db33eeb2034681bdd94bdeb2190926 (127.21.55.65). New cstate: current_term: 1 leader_uuid: "07db33eeb2034681bdd94bdeb2190926" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "395a500db03846d98ecf72fbfe98bbbe" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07db33eeb2034681bdd94bdeb2190926" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89908f912516448c8f6fc0c1a7a4d165" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39405 } health_report { overall_health: UNKNOWN } } }
W20260810 14:39:11.861471 6746 tablet.cc:2369] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:39:11.866645 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.860583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.21.55.65:33733)
0810 14:39:11.860816 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:11.860823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:11.860844 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:39:11.860985 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:11.860989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:11.861104 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:11.861284 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:39:11.861750 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.862740 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.864455 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.864842 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.864936 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.865496 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:39:11.866006 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:39:11.866132 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:11.866257 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 14:39:11.866323 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:39:11.874799 7114 raft_consensus.cc:1261] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe [term 1 FOLLOWER]: Refusing update from remote peer 07db33eeb2034681bdd94bdeb2190926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:39:11.874800 6895 raft_consensus.cc:1261] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165 [term 1 FOLLOWER]: Refusing update from remote peer 07db33eeb2034681bdd94bdeb2190926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:39:11.875855 7234 consensus_queue.cc:1035] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "395a500db03846d98ecf72fbfe98bbbe" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:39:11.876155 7246 consensus_queue.cc:1035] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89908f912516448c8f6fc0c1a7a4d165" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:39:11.906088 6434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:39:11.906520 6434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:39:11.910655 6434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de5d51540cc84e39a6b51fd2d1ca8884: Sending DeleteTablet for 3 replicas of tablet f14f519e7c994adfbe67a68bdad6f889
I20260810 14:39:11.911872 7081 tablet_service.cc:1514] Processing DeleteTablet for tablet f14f519e7c994adfbe67a68bdad6f889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:39:11 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:46758
I20260810 14:39:11.911897 6642 tablet_service.cc:1514] Processing DeleteTablet for tablet f14f519e7c994adfbe67a68bdad6f889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:39:11 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:60912
I20260810 14:39:11.912667 6868 tablet_service.cc:1514] Processing DeleteTablet for tablet f14f519e7c994adfbe67a68bdad6f889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:39:11 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:44656
I20260810 14:39:11.913664 7270 tablet_replica.cc:330] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: stopping tablet replica
I20260810 14:39:11.915266 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 6542
I20260810 14:39:11.923982 7270 raft_consensus.cc:2229] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:39:11.924485 7268 tablet_replica.cc:330] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: stopping tablet replica
I20260810 14:39:11.924906 7270 raft_consensus.cc:2258] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:39:11.925206 7268 raft_consensus.cc:2229] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:39:11.925837 7268 raft_consensus.cc:2258] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:39:11.926992 7267 tablet_replica.cc:330] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: stopping tablet replica
I20260810 14:39:11.928028 7270 ts_tablet_manager.cc:1905] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:39:11.928457 7268 ts_tablet_manager.cc:1905] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:39:11.935281 7267 raft_consensus.cc:2229] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:39:11.935485 7270 ts_tablet_manager.cc:1918] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:39:11.935714 7270 log.cc:1196] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/f14f519e7c994adfbe67a68bdad6f889
I20260810 14:39:11.936376 7267 raft_consensus.cc:2258] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:39:11.936425 7270 ts_tablet_manager.cc:1939] T f14f519e7c994adfbe67a68bdad6f889 P 89908f912516448c8f6fc0c1a7a4d165: Deleting consensus metadata
I20260810 14:39:11.936662 7268 ts_tablet_manager.cc:1918] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:39:11.936880 7268 log.cc:1196] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/f14f519e7c994adfbe67a68bdad6f889
I20260810 14:39:11.937474 7268 ts_tablet_manager.cc:1939] T f14f519e7c994adfbe67a68bdad6f889 P 395a500db03846d98ecf72fbfe98bbbe: Deleting consensus metadata
I20260810 14:39:11.938517 6416 catalog_manager.cc:4841] TS 89908f912516448c8f6fc0c1a7a4d165 (127.21.55.66:39405): tablet f14f519e7c994adfbe67a68bdad6f889 (table test-table [id=ab819ea42e314eb7bfe6993a5ecf7245]) successfully deleted
I20260810 14:39:11.939213 6412 catalog_manager.cc:4841] TS 395a500db03846d98ecf72fbfe98bbbe (127.21.55.67:44129): tablet f14f519e7c994adfbe67a68bdad6f889 (table test-table [id=ab819ea42e314eb7bfe6993a5ecf7245]) successfully deleted
Aug 10 14:39:11 dist-test-slave-h6n0 krb5kdc[6352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:39:11.940843 7267 ts_tablet_manager.cc:1905] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:39:11.948382 7267 ts_tablet_manager.cc:1918] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:39:11.948630 7267 log.cc:1196] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/f14f519e7c994adfbe67a68bdad6f889
I20260810 14:39:11.949314 7267 ts_tablet_manager.cc:1939] T f14f519e7c994adfbe67a68bdad6f889 P 07db33eeb2034681bdd94bdeb2190926: Deleting consensus metadata
I20260810 14:39:11.949338 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:11.916790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60942 (local address 127.21.55.65:33733)
0810 14:39:11.916977 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:11.916983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:11.927228 (+ 10245us) server_negotiation.cc:407] Connection header received
0810 14:39:11.927545 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:11.927552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:11.927645 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:11.927729 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:11.929645 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.930852 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.932725 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:11.933812 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:11.933935 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:11.941241 (+ 7306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:11.941253 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:11.941268 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:39:11.941324 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:11.945716 (+ 4392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:11.946682 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:11.946691 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:11.946695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:11.946887 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:11.947355 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:11.947361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:11.947364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:11.947819 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:11.948176 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:11.948545 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 14:39:11.948871 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:39:11.950151 6739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:12.052781 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 6542
I20260810 14:39:12.052825 6417 catalog_manager.cc:4841] TS 07db33eeb2034681bdd94bdeb2190926 (127.21.55.65:33733): tablet f14f519e7c994adfbe67a68bdad6f889 (table test-table [id=ab819ea42e314eb7bfe6993a5ecf7245]) successfully deleted
I20260810 14:39:12.081946 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 6754
Aug 10 14:39:12 dist-test-slave-h6n0 krb5kdc[6352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:39:12.097505 7215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:12.082806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44660 (local address 127.21.55.66:39405)
0810 14:39:12.083002 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:12.083008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:12.083029 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:39:12.083194 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:12.083199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:12.083287 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:12.083397 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:12.084693 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:12.085507 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:12.086893 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:12.087276 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:12.087406 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:12.091755 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:12.091768 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:12.091792 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:39:12.091830 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:12.094656 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:12.095332 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:12.095339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:12.095342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:12.095471 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:12.095867 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:12.095871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:12.095873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:12.096242 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:12.096571 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:12.096906 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 14:39:12.097125 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:39:12.098146 6962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:12.188684 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 6754
I20260810 14:39:12.217141 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 6982
Aug 10 14:39:12 dist-test-slave-h6n0 krb5kdc[6352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:39:12.232690 7212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:39:12.218117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.21.55.67:44129)
0810 14:39:12.218323 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 14:39:12.218328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:39:12.218346 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:39:12.218421 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:39:12.218425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:39:12.218512 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:39:12.218646 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:39:12.220152 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:12.220876 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:12.222124 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:39:12.222511 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:39:12.222829 (+ 318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:39:12.227076 (+ 4247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:39:12.227091 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:39:12.227109 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:39:12.227142 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:39:12.230008 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:12.230918 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:12.230925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:12.230927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:12.231082 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:39:12.231377 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:39:12.231380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:39:12.231382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:39:12.231713 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:39:12.231942 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 14:39:12.232169 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 14:39:12.232328 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 14:39:12.233340 7180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:12.330876 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 6982
I20260810 14:39:12.363302 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 6375
I20260810 14:39:12.365091 6487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:39:12.490001 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 6375
2026-08-10T14:39:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5218 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:39:12 dist-test-slave-h6n0 krb5kdc[7283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:39:12 dist-test-slave-h6n0 krb5kdc[7283](info): set up 2 sockets
Aug 10 14:39:12 dist-test-slave-h6n0 krb5kdc[7283](info): commencing operation
krb5kdc: starting...
W20260810 14:39:13.923450 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.375s user 0.005s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:39:14 dist-test-slave-h6n0 krb5kdc[7283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:39:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:39:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.55.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.55.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/rangeradmin_127.21.55.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/rangeradmin_127.21.55.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.55.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.55.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/rangerlookup_127.21.55.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/rangerlookup_127.21.55.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.55.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.55.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/HTTP_127.21.55.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/HTTP_127.21.55.84@KRBTEST.COM.keytab.
I20260810 14:39:14.397745 21725 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwtLR1L/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/postgres -l logfile start
2026-08-10 14:39:17.107 UTC [7332] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 14:39:17.108 UTC [7332] LOG: listening on IPv4 address "127.21.55.84", port 49253
2026-08-10 14:39:17.111 UTC [7332] LOG: listening on Unix socket "/tmp/.s.PGSQL.49253"
2026-08-10 14:39:17.136 UTC [7334] LOG: database system was shut down at 2026-08-10 14:39:15 UTC
2026-08-10 14:39:17.140 UTC [7332] LOG: database system is ready to accept connections
I20260810 14:39:18.408172 21725 mini_postgres.cc:96] Postgres bound to 49253
2026-08-10 14:39:18.436 UTC [7349] FATAL: database "slave" does not exist
127.21.55.84:49253 - accepting connections
I20260810 14:39:18.437263 21725 mini_ranger.cc:161] Starting Ranger...
I20260810 14:39:18.474699 21725 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 14:39:19.016708 21725 mini_ranger.cc:87] Created ranger Postgres database
I20260810 14:39:19.016855 21725 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-admin
2026-08-10 14:39:19,324 [I] DB FLAVOR :POSTGRES
2026-08-10 14:39:19,325 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:39:19,325 [I] Checking connection..
2026-08-10 14:39:19,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 14:39:19,609 [I] Checking connection passed.
2026-08-10 14:39:19,609 [I] --------- Verifying version history table ---------
2026-08-10 14:39:19,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:39:19,899 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:39:19,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:39:20,189 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:39:20,189 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 14:39:20,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 14:39:20.451 UTC [7451] WARNING: there is no transaction in progress
2026-08-10 14:39:20,460 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 14:39:20,461 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:39:20,745 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 14:39:20,746 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:39:20,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 14:39:21,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 14:39:21,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h6n0.c.gcp-upstream.internal','N') ;"
2026-08-10 14:39:21,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:39:21,766 [I] Table x_portal_user does not exist in database ranger
2026-08-10 14:39:21,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:39:22,053 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 14:39:22,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:39:22,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:39:22,586 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 14:39:22,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 14:39:23.607 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.627 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.647 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.665 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.684 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.702 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.765 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.773 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.790 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.801 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.814 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.830 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.840 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.847 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.855 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:23.862 UTC [7628] WARNING: there is no transaction in progress
2026-08-10 14:39:24,584 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 14:39:24,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 14:39:24,835 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 14:39:24,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:39:25,094 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 14:39:25,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:39:25,377 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 14:39:25,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:39:25,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:39:25,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h6n0.c.gcp-upstream.internal';"
2026-08-10 14:39:26,221 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 14:39:26,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 14:39:26,516 [I] DB_PATCHES have already been applied
I20260810 14:39:26.523939 21725 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwtLR1L/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 14:39:26.524045 21725 mini_ranger.cc:193] Using host: 127.21.55.84
I20260810 14:39:26.548460 21725 mini_ranger.cc:227] Ranger admin URL: http://127.21.55.84:38075
Aug 10, 2026 2:39:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:39:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:39:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 2:39:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 2:39:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 2:39:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 2:39:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 14:39:28.083390 21725 mini_ranger.cc:160] Time spent starting Ranger: real 9.646s user 0.004s sys 0.064s
Aug 10, 2026 2:39:49 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 2:39:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 10, 2026 2:39:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 2:39:49 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 2:39:50 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 14:39:51.219077 21725 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.55.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.55.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/rangerkms_127.21.55.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/rangerkms_127.21.55.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.55.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.55.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.55.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/HTTP_127.21.55.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.55.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/HTTP_127.21.55.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 14:39:51 dist-test-slave-h6n0 krb5kdc[7283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372791, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 14:39:51.446302 21725 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 14:39:51.478010 21725 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 14:39:52.474155 21725 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 14:39:52.474359 21725 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-kms
I20260810 14:39:52.474501 21725 mini_ranger_kms.cc:227] Using postgres at 127.21.55.84:49253
2026-08-10 14:39:53,392 [I] DB FLAVOR :POSTGRES
2026-08-10 14:39:53,392 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:39:53,392 [I] Checking connection
2026-08-10 14:39:53,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 14:39:53,659 [I] connection success
2026-08-10 14:39:53,659 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 14:39:53,659 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 14:39:53,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 14:39:53,904 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 14:39:53,904 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:39:53,904 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 14:39:53,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:49253/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 14:39:54,176 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 14:39:54.667722 21725 mini_ranger_kms.cc:310] Created kms service
I20260810 14:39:54.880522 21725 mini_ranger_kms.cc:326] Created kudu user
I20260810 14:39:54.925623 21725 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 14:39:54 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372791, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:39:55.344496 21725 mini_ranger_kms.cc:384] Added ranger policy
I20260810 14:39:55.346077 21725 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 14:39:55.346184 21725 mini_ranger_kms.cc:240] Using host: 127.21.55.84
I20260810 14:39:55.372537 21725 mini_ranger_kms.cc:276] Ranger KMS PID: 7913
I20260810 14:39:55.372776 21725 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.55.84:55819
Aug 10, 2026 2:39:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:39:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:39:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 2:39:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 2:39:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 2:39:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 14:39:57 dist-test-slave-h6n0 krb5kdc[7283](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786372797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.55.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 14:39:57 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.55.84@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:39:58.334070 21725 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.888s user 0.146s sys 0.492s
I20260810 14:39:58.334440 21725 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 14:39:58.334823 21725 mini_ranger_kms.cc:401] 127.21.55.84:55819/kms/v1/keys
Aug 10 14:39:58 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.55.84@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
Aug 10 14:40:00 dist-test-slave-h6n0 krb5kdc[7283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:40:00.976738 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:36895
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:36895
--ranger_config_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:55819/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:40:01.372205 7995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:40:01.372511 7995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:40:01.372613 7995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:40:01.380714 7995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:40:01.381049 7995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:40:01.381196 7995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:40:01.381265 7995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:40:01.381469 7995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:40:01.392201 7995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.55.84:55819/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:36895
--ranger_config_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:36895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:40:01.393566 7995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:40:01.394831 7995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:40:01.407104 8009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:01.407423 8007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:01.407491 8014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:40:01.408329 7995 server_base.cc:1034] running on GCE node
I20260810 14:40:01.409194 7995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:40:01.411075 7995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:40:01.412292 7995 hybrid_clock.cc:648] HybridClock initialized: now 1786372801412250 us; error 43 us; skew 500 ppm
I20260810 14:40:01.412703 7995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:40:01 dist-test-slave-h6n0 krb5kdc[7283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372801, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:40:01.420997 7995 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:40:01.422801 7995 webserver.cc:460] Webserver started at http://127.21.55.126:33607/ using document root <none> and password file <none>
I20260810 14:40:01.423512 7995 fs_manager.cc:359] Metadata directory not provided
I20260810 14:40:01.423643 7995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:40:01.423992 7995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:40:01 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372801, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:40:01.492338 7995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "4a2734547ef648f59e016a2fe27236b4"
format_stamp: "Formatted at 2026-08-10 14:40:01 on dist-test-slave-h6n0"
server_key: "af059121efe978f2a2a75a10329c2084"
server_key_iv: "e16942a67078bde4443f7aab1662ac5f"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:01.493495 7995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "4a2734547ef648f59e016a2fe27236b4"
format_stamp: "Formatted at 2026-08-10 14:40:01 on dist-test-slave-h6n0"
server_key: "af059121efe978f2a2a75a10329c2084"
server_key_iv: "e16942a67078bde4443f7aab1662ac5f"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:01.499872 7995 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 14:40:01.522922 8029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:01.525301 7995 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260810 14:40:01.525688 7995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "4a2734547ef648f59e016a2fe27236b4"
format_stamp: "Formatted at 2026-08-10 14:40:01 on dist-test-slave-h6n0"
server_key: "af059121efe978f2a2a75a10329c2084"
server_key_iv: "e16942a67078bde4443f7aab1662ac5f"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:01.525988 7995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:40:01.543751 7995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:40:01.548839 7995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:40:01.549221 7995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:40:01.571151 7995 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:36895
I20260810 14:40:01.571192 8125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:36895 every 8 connection(s)
I20260810 14:40:01.572606 7995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:40:01.577448 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 7995
I20260810 14:40:01.577883 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:40:01.578064 8126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:40:01.579164 21725 external_mini_cluster.cc:1428] Setting key 852fbb0bc5c352d8888d703a18b60aae
I20260810 14:40:01.595633 8126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4: Bootstrap starting.
Aug 10 14:40:01 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:40:01.599164 8126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4: Neither blocks nor log segments found. Creating new log.
I20260810 14:40:01.600468 8126 log.cc:824] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4: Log is configured to *not* fsync() on all Append() calls
I20260810 14:40:01.604331 8126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4: No bootstrap required, opened a new log
I20260810 14:40:01.609855 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:01.582614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51932 (local address 127.21.55.126:36895)
0810 14:40:01.583791 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:01.583805 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:01.583856 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:40:01.586994 (+ 3138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:01.587021 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:01.587481 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:01.587867 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:40:01.589662 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:01.591866 (+ 2204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:01.593913 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:01.594851 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:01.595014 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:01.599673 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:40:01.599682 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:40:01.599703 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:40:01.599754 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:40:01.604098 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:01.605342 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:01.605351 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:01.605361 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:01.605619 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:01.606480 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:01.606502 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:01.606505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:01.607249 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:40:01.607627 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:01.608216 (+ 589us) server_negotiation.cc:299] Negotiation successful
0810 14:40:01.608726 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":887,"thread_start_us":564,"threads_started":1}
I20260810 14:40:01.612389 8126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2734547ef648f59e016a2fe27236b4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36895 } }
I20260810 14:40:01.612869 8126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:40:01.612994 8126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a2734547ef648f59e016a2fe27236b4, State: Initialized, Role: FOLLOWER
I20260810 14:40:01.613574 8126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2734547ef648f59e016a2fe27236b4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36895 } }
I20260810 14:40:01.613797 8126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:40:01.613895 8126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:40:01.614022 8126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:40:01.617205 8126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2734547ef648f59e016a2fe27236b4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36895 } }
I20260810 14:40:01.617820 8126 leader_election.cc:302] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a2734547ef648f59e016a2fe27236b4; no voters:
I20260810 14:40:01.618505 8126 leader_election.cc:288] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:40:01.618660 8132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:40:01.619833 8132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [term 1 LEADER]: Becoming Leader. State: Replica: 4a2734547ef648f59e016a2fe27236b4, State: Running, Role: LEADER
I20260810 14:40:01.620498 8132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2734547ef648f59e016a2fe27236b4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36895 } }
I20260810 14:40:01.621368 8126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:40:01.624413 8133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a2734547ef648f59e016a2fe27236b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2734547ef648f59e016a2fe27236b4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36895 } } }
I20260810 14:40:01.624549 8139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a2734547ef648f59e016a2fe27236b4. Latest consensus state: current_term: 1 leader_uuid: "4a2734547ef648f59e016a2fe27236b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a2734547ef648f59e016a2fe27236b4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36895 } } }
I20260810 14:40:01.624876 8133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:40:01.624953 8139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:40:01.626086 8149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:40:01.633417 8149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:40:01.640594 8126 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 14:40:01.653012 8149 catalog_manager.cc:1347] Generated new cluster ID: 12430670d6a641cf8007fb98bc89edeb
I20260810 14:40:01.653353 8149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:40:01.691927 8149 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:40:01.694092 8149 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:40:01.704942 8149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4: Generated new TSK 0
I20260810 14:40:01.706132 8149 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:40:02.526752 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:55819/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:36895
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:40:02 dist-test-slave-h6n0 krb5kdc[7283](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786372802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 14:40:03.021502 8211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:40:03.021945 8211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:40:03.022111 8211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:40:03.034134 8211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:40:03.034368 8211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:40:03.034616 8211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:40:03.050911 8211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.55.84:55819/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:36895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:40:03.052697 8211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:40:03.054563 8211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:40:03.075101 8240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:03.076102 8241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:03.077900 8244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:40:03.079905 8211 server_base.cc:1034] running on GCE node
I20260810 14:40:03.080657 8211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:40:03.081781 8211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:40:03.083045 8211 hybrid_clock.cc:648] HybridClock initialized: now 1786372803082984 us; error 53 us; skew 500 ppm
I20260810 14:40:03.083468 8211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:40:03 dist-test-slave-h6n0 krb5kdc[7283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372803, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:40:03.087270 8211 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:40:03.089138 8211 webserver.cc:460] Webserver started at http://127.21.55.65:40221/ using document root <none> and password file <none>
I20260810 14:40:03.090006 8211 fs_manager.cc:359] Metadata directory not provided
I20260810 14:40:03.090174 8211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:40:03.090602 8211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:40:03 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372803, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:40:03.111907 8211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "dace796d542945c7975fd1861594f2c6"
format_stamp: "Formatted at 2026-08-10 14:40:03 on dist-test-slave-h6n0"
server_key: "644951f5d0e5c917c9628927ad4d2912"
server_key_iv: "dcdb3df5e80fde555687f3dc3e74214b"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:03.112951 8211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "dace796d542945c7975fd1861594f2c6"
format_stamp: "Formatted at 2026-08-10 14:40:03 on dist-test-slave-h6n0"
server_key: "644951f5d0e5c917c9628927ad4d2912"
server_key_iv: "dcdb3df5e80fde555687f3dc3e74214b"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:03.120698 8211 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260810 14:40:03.155854 8252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:03.164300 8211 fs_manager.cc:725] Time spent opening block manager: real 0.024s user 0.001s sys 0.015s
I20260810 14:40:03.164557 8211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "dace796d542945c7975fd1861594f2c6"
format_stamp: "Formatted at 2026-08-10 14:40:03 on dist-test-slave-h6n0"
server_key: "644951f5d0e5c917c9628927ad4d2912"
server_key_iv: "dcdb3df5e80fde555687f3dc3e74214b"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:03.164878 8211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:40:03.211934 8211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:40:03.217684 8211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:40:03.218142 8211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:40:03.219929 8211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:40:03.231413 8211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:40:03.231632 8211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:03.231861 8211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:40:03.232038 8211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:40:03 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:40:03.419279 8211 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:42791
I20260810 14:40:03.419227 8438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:42791 every 8 connection(s)
I20260810 14:40:03.421113 8211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:40:03.422832 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 8211
I20260810 14:40:03.423085 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:40:03.424191 21725 external_mini_cluster.cc:1428] Setting key 4e637bdffacfe33de348a30d87670338
Aug 10 14:40:03 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372803, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:40:03.468377 8441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:03.426104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:50225 (local address 127.21.55.126:36895)
0810 14:40:03.435114 (+ 9010us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:03.435121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:03.441042 (+ 5921us) server_negotiation.cc:407] Connection header received
0810 14:40:03.445442 (+ 4400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:03.445449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:03.445543 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:03.445672 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:40:03.448664 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:03.450152 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:03.453262 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:03.454050 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:03.454168 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:03.459386 (+ 5218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:40:03.459393 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:40:03.459398 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:40:03.459438 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:40:03.463882 (+ 4444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:03.465040 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:03.465046 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:03.465050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:03.465210 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:03.465865 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:03.465871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:03.465874 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:03.466242 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:40:03.466372 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:03.467576 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0810 14:40:03.467882 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8867,"thread_start_us":8672,"threads_started":1}
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
I20260810 14:40:03.471251 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:03.426646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:36895 (local address 127.21.55.65:50225)
0810 14:40:03.440845 (+ 14199us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:03.440901 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 14:40:03.445039 (+ 4138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:40:03.446168 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:40:03.446182 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:40:03.447101 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:40:03.448435 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:03.448463 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:03.450443 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:03.450449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:03.453077 (+ 2628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:03.453094 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:03.454699 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:03.454710 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:03.455096 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:03.455990 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:40:03.456021 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:40:03.459110 (+ 3089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:40:03.464105 (+ 4995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:40:03.464115 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:40:03.464134 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:40:03.464887 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:40:03.465355 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:40:03.465359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:40:03.465363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:40:03.465692 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:40:03.466519 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:40:03.466546 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:40:03.467041 (+ 495us) client_negotiation.cc:769] Sending connection context
0810 14:40:03.468867 (+ 1826us) client_negotiation.cc:240] Negotiation successful
0810 14:40:03.469441 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13831,"thread_start_us":14296,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
I20260810 14:40:03.473803 8439 heartbeater.cc:347] Connected to a master server at 127.21.55.126:36895
I20260810 14:40:03.474210 8439 heartbeater.cc:464] Registering TS with master...
I20260810 14:40:03.475396 8439 heartbeater.cc:511] Master 127.21.55.126:36895 requested a full tablet report, sending...
I20260810 14:40:03.478014 8058 ts_manager.cc:194] Registered new tserver with Master: dace796d542945c7975fd1861594f2c6 (127.21.55.65:42791)
I20260810 14:40:03.481051 8058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:50225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:40:03.524600 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:55819/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:36895
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:40:03 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372802, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
W20260810 14:40:04.169178 8453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:40:04.169513 8453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:40:04.169613 8453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:40:04.182817 8453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:40:04.183228 8453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:40:04.183590 8453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:40:04.203032 8453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.55.84:55819/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:36895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:40:04.204744 8453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:40:04.206372 8453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:40:04.228057 8477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:04.228130 8479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:04.229008 8476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:40:04.236460 8453 server_base.cc:1034] running on GCE node
I20260810 14:40:04.237329 8453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:40:04.238744 8453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:40:04.240254 8453 hybrid_clock.cc:648] HybridClock initialized: now 1786372804240060 us; error 121 us; skew 500 ppm
I20260810 14:40:04.240748 8453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:40:04 dist-test-slave-h6n0 krb5kdc[7283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372804, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:40:04.246407 8453 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:40:04.248621 8453 webserver.cc:460] Webserver started at http://127.21.55.66:44499/ using document root <none> and password file <none>
I20260810 14:40:04.249383 8453 fs_manager.cc:359] Metadata directory not provided
I20260810 14:40:04.249521 8453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:40:04.249862 8453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:40:04 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372804, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:40:04.272676 8453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "051641f4857b404a98d7dcce5376b57a"
format_stamp: "Formatted at 2026-08-10 14:40:04 on dist-test-slave-h6n0"
server_key: "c81efdea5d12c884cd09615ec81e08df"
server_key_iv: "0cbfca1c61ca0e1ad556f0001cf48823"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:04.274370 8453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "051641f4857b404a98d7dcce5376b57a"
format_stamp: "Formatted at 2026-08-10 14:40:04 on dist-test-slave-h6n0"
server_key: "c81efdea5d12c884cd09615ec81e08df"
server_key_iv: "0cbfca1c61ca0e1ad556f0001cf48823"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:04.298710 8453 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.018s sys 0.003s
I20260810 14:40:04.325182 8495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:04.327587 8453 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260810 14:40:04.327824 8453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "051641f4857b404a98d7dcce5376b57a"
format_stamp: "Formatted at 2026-08-10 14:40:04 on dist-test-slave-h6n0"
server_key: "c81efdea5d12c884cd09615ec81e08df"
server_key_iv: "0cbfca1c61ca0e1ad556f0001cf48823"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:04.328372 8453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:40:04.353085 8453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:40:04.359123 8453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:40:04.359535 8453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:40:04.361111 8453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:40:04.363790 8453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:40:04.363986 8453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:04.364240 8453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:40:04.364396 8453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:04.459431 8453 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:34495
I20260810 14:40:04.459635 8677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:34495 every 8 connection(s)
I20260810 14:40:04.461035 8453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:40:04.464290 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 8453
I20260810 14:40:04.464514 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:40:04.465276 21725 external_mini_cluster.cc:1428] Setting key e234d7c07738e2aee7234b74e23422f5
I20260810 14:40:04.487723 8439 heartbeater.cc:503] Master 127.21.55.126:36895 was elected leader, sending a full tablet report...
Aug 10 14:40:04 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372804, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:40:04.497587 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:04.464873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:39799 (local address 127.21.55.126:36895)
0810 14:40:04.473041 (+ 8168us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:04.473049 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:04.473073 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:40:04.473138 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:04.473145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:04.473294 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:04.473432 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:40:04.476536 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:04.478150 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:04.481523 (+ 3373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:04.482499 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:04.482802 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:04.489347 (+ 6545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:40:04.489355 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:40:04.489360 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:40:04.489398 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:40:04.493395 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:04.494212 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:04.494219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:04.494228 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:04.494407 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:04.495019 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:04.495024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:04.495026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:04.495636 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:40:04.495800 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:04.496679 (+ 879us) server_negotiation.cc:299] Negotiation successful
0810 14:40:04.496968 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8012,"thread_start_us":7779,"threads_started":1}
I20260810 14:40:04.498788 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:04.465905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:36895 (local address 127.21.55.66:39799)
0810 14:40:04.467017 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:04.467082 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 14:40:04.470158 (+ 3076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:40:04.473830 (+ 3672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:40:04.473846 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:40:04.474980 (+ 1134us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:40:04.476286 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:04.476320 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:04.478343 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:04.478351 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:04.481203 (+ 2852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:04.481223 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:04.482886 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:04.482896 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:04.483288 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:04.484517 (+ 1229us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:40:04.484576 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:40:04.489040 (+ 4464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:40:04.493565 (+ 4525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:40:04.493576 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:40:04.493595 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:40:04.494073 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:40:04.494571 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:40:04.494576 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:40:04.494580 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:40:04.494878 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:40:04.495779 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:40:04.495788 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:40:04.496234 (+ 446us) client_negotiation.cc:769] Sending connection context
0810 14:40:04.496525 (+ 291us) client_negotiation.cc:240] Negotiation successful
0810 14:40:04.497140 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":5093,"threads_started":1}
I20260810 14:40:04.501117 8680 heartbeater.cc:347] Connected to a master server at 127.21.55.126:36895
I20260810 14:40:04.501597 8680 heartbeater.cc:464] Registering TS with master...
I20260810 14:40:04.502933 8680 heartbeater.cc:511] Master 127.21.55.126:36895 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
I20260810 14:40:04.504930 8058 ts_manager.cc:194] Registered new tserver with Master: 051641f4857b404a98d7dcce5376b57a (127.21.55.66:34495)
I20260810 14:40:04.505944 8058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:39799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:40:04.551465 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:55819/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:36895
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:40815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:40:05.092967 8693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:40:05.093315 8693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:40:05.093492 8693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:40:05.102701 8693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:40:05.102862 8693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:40:05.103056 8693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:40:05.113780 8693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:40815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.55.84:55819/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:36895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:40:05.115197 8693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:40:05.116530 8693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:40:05.133230 8698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:05.133920 8710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:05.133973 8700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:40:05.135358 8693 server_base.cc:1034] running on GCE node
I20260810 14:40:05.136292 8693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:40:05.137742 8693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:40:05.139161 8693 hybrid_clock.cc:648] HybridClock initialized: now 1786372805139061 us; error 69 us; skew 500 ppm
I20260810 14:40:05.139843 8693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:40:05 dist-test-slave-h6n0 krb5kdc[7283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372805, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:40:05.144215 8693 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:40:05.146021 8693 webserver.cc:460] Webserver started at http://127.21.55.67:44979/ using document root <none> and password file <none>
I20260810 14:40:05.146766 8693 fs_manager.cc:359] Metadata directory not provided
I20260810 14:40:05.146876 8693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:40:05.147135 8693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:40:05 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372805, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:40:05.165093 8693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "cee78e0796604ad6bd98a67c6c4ce4f1"
format_stamp: "Formatted at 2026-08-10 14:40:05 on dist-test-slave-h6n0"
server_key: "d90268f082910cf5862b3af9a211786f"
server_key_iv: "f5784a1678ce9eafe2f6124970790f38"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:05.166246 8693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "cee78e0796604ad6bd98a67c6c4ce4f1"
format_stamp: "Formatted at 2026-08-10 14:40:05 on dist-test-slave-h6n0"
server_key: "d90268f082910cf5862b3af9a211786f"
server_key_iv: "f5784a1678ce9eafe2f6124970790f38"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:05.172765 8693 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 14:40:05.193120 8725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:05.195298 8693 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.004s
I20260810 14:40:05.195536 8693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "cee78e0796604ad6bd98a67c6c4ce4f1"
format_stamp: "Formatted at 2026-08-10 14:40:05 on dist-test-slave-h6n0"
server_key: "d90268f082910cf5862b3af9a211786f"
server_key_iv: "f5784a1678ce9eafe2f6124970790f38"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:05.195779 8693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:40:05.223263 8693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:40:05.227903 8693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:40:05.228269 8693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:40:05.229624 8693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:40:05.231987 8693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:40:05.232067 8693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:05.232218 8693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:40:05.232273 8693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:05.299275 8693 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:44955
I20260810 14:40:05.299331 8902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:44955 every 8 connection(s)
I20260810 14:40:05.301889 8693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:40:05.307405 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 8693
I20260810 14:40:05.307652 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:40:05.308274 21725 external_mini_cluster.cc:1428] Setting key f32842daa8bb26dfac0110d3883b5245
Aug 10 14:40:05 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372805, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:40:05.338655 8914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.307213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:52143 (local address 127.21.55.126:36895)
0810 14:40:05.315281 (+ 8068us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:05.315286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:05.315305 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:40:05.315375 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:05.315378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:05.315461 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:05.315566 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:40:05.317274 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.318247 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.320690 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.321296 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.321410 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.327145 (+ 5735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:40:05.327150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:40:05.327153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:40:05.327191 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:40:05.332131 (+ 4940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:05.333843 (+ 1712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:05.333848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:05.333851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:05.334304 (+ 453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:05.335428 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:05.335434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:05.335437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:05.336282 (+ 845us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:40:05.336547 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:05.337640 (+ 1093us) server_negotiation.cc:299] Negotiation successful
0810 14:40:05.338096 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7946,"thread_start_us":7735,"threads_started":1}
I20260810 14:40:05.340310 8909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.306995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:36895 (local address 127.21.55.67:52143)
0810 14:40:05.309037 (+ 2042us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:05.309161 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0810 14:40:05.314506 (+ 5345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:40:05.315889 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:40:05.315900 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:40:05.316440 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:40:05.317073 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.317091 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.318617 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.318631 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.320504 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.320522 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.321678 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.321686 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.322155 (+ 469us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.323327 (+ 1172us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:40:05.323357 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:40:05.326879 (+ 3522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:40:05.332480 (+ 5601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:40:05.332497 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:40:05.332525 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:40:05.333467 (+ 942us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:40:05.334611 (+ 1144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:40:05.334618 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:40:05.334622 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:40:05.335161 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:40:05.336466 (+ 1305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:40:05.336476 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:40:05.337048 (+ 572us) client_negotiation.cc:769] Sending connection context
0810 14:40:05.337489 (+ 441us) client_negotiation.cc:240] Negotiation successful
0810 14:40:05.338228 (+ 739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1438,"thread_start_us":812,"threads_started":1}
I20260810 14:40:05.342423 8903 heartbeater.cc:347] Connected to a master server at 127.21.55.126:36895
I20260810 14:40:05.343041 8903 heartbeater.cc:464] Registering TS with master...
I20260810 14:40:05.344149 8903 heartbeater.cc:511] Master 127.21.55.126:36895 requested a full tablet report, sending...
I20260810 14:40:05.346678 8058 ts_manager.cc:194] Registered new tserver with Master: cee78e0796604ad6bd98a67c6c4ce4f1 (127.21.55.67:44955)
I20260810 14:40:05.347641 8058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:52143
I20260810 14:40:05.354254 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:40:05.378682 8914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.361260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51942 (local address 127.21.55.126:36895)
0810 14:40:05.361471 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:05.361476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:05.362687 (+ 1211us) server_negotiation.cc:407] Connection header received
0810 14:40:05.362998 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:05.363003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:05.363086 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:05.363202 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:40:05.365059 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.365978 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.367838 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.368290 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.368443 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.371227 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:40:05.371232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:40:05.371236 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:40:05.371269 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:40:05.375446 (+ 4177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:05.376334 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:05.376340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:05.376342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:05.376493 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:05.376949 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:05.376957 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:05.376962 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:05.377279 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:40:05.377596 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:05.377938 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 14:40:05.378213 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 14:40:05.383721 8058 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:40:05.393280 8058 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:40:05.444314 8933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.410945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.21.55.66:34495)
0810 14:40:05.430604 (+ 19659us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:05.430620 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:05.430662 (+ 42us) server_negotiation.cc:407] Connection header received
0810 14:40:05.430855 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:05.430868 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:05.431318 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:05.431689 (+ 371us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:40:05.435038 (+ 3349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.437149 (+ 2111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.441024 (+ 3875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.442983 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.443201 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.443473 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:05.443924 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 14:40:05.444055 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19384,"thread_start_us":19155,"threads_started":1}
I20260810 14:40:05.444692 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.411446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:34495 (local address 127.0.0.1:52264)
0810 14:40:05.429093 (+ 17647us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:05.429149 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 14:40:05.429370 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:40:05.432311 (+ 2941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:40:05.432324 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:40:05.432373 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:40:05.433791 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.433814 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.437399 (+ 3585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.437405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.440840 (+ 3435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.440852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.443416 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.443425 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.443668 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.443706 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 14:40:05.443879 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 14:40:05.444130 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17300,"mutex_wait_us":26,"thread_start_us":23207,"threads_started":1}
I20260810 14:40:05.446846 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.417941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42791 (local address 127.0.0.1:38240)
0810 14:40:05.435080 (+ 17139us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:05.435102 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:40:05.435315 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:40:05.436451 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:40:05.436456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:40:05.436482 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:40:05.437414 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.437430 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.439469 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.439474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.443649 (+ 4175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.443664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.445831 (+ 2167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.445837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.446016 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.446040 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 14:40:05.446156 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 14:40:05.446348 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16930,"mutex_wait_us":231,"thread_start_us":15668,"threads_started":1}
I20260810 14:40:05.447221 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.418141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.21.55.65:42791)
0810 14:40:05.432982 (+ 14841us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:05.432993 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:05.435204 (+ 2211us) server_negotiation.cc:407] Connection header received
0810 14:40:05.435600 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:05.435611 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:05.435843 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:05.436051 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:40:05.437670 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.439300 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.443834 (+ 4534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.445607 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.445720 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.446014 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:05.446255 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 14:40:05.446443 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14566,"thread_start_us":14813,"threads_started":1}
I20260810 14:40:05.449823 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.411652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:44955 (local address 127.0.0.1:48080)
0810 14:40:05.434181 (+ 22529us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:05.434202 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:40:05.434376 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:40:05.435784 (+ 1408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:40:05.435789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:40:05.435816 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:40:05.436467 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.436483 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.439235 (+ 2752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.439240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.443663 (+ 4423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.443673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.448858 (+ 5185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.448868 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.449093 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.449127 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 14:40:05.449239 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 14:40:05.449376 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22401,"mutex_wait_us":153,"thread_start_us":17602,"threads_started":1}
I20260810 14:40:05.449569 8577 tablet_service.cc:1467] Processing CreateTablet for tablet e45134135987459e9da32edcd58dece6 (DEFAULT_TABLE table=test-table [id=2e15ecbf86924062a71b40c15dda045d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:40:05.450258 8947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.412521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48080 (local address 127.21.55.67:44955)
0810 14:40:05.434914 (+ 22393us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:05.434923 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:05.434978 (+ 55us) server_negotiation.cc:407] Connection header received
0810 14:40:05.435086 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:05.435094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:05.435525 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:05.435782 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:40:05.436664 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.438975 (+ 2311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.446155 (+ 7180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.448163 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.448305 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.448610 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:05.449353 (+ 743us) server_negotiation.cc:299] Negotiation successful
0810 14:40:05.449644 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22150,"thread_start_us":21757,"threads_started":1}
I20260810 14:40:05.451562 8577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e45134135987459e9da32edcd58dece6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:40:05.451481 8324 tablet_service.cc:1467] Processing CreateTablet for tablet e45134135987459e9da32edcd58dece6 (DEFAULT_TABLE table=test-table [id=2e15ecbf86924062a71b40c15dda045d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:40:05.452976 8324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e45134135987459e9da32edcd58dece6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:40:05.459103 8805 tablet_service.cc:1467] Processing CreateTablet for tablet e45134135987459e9da32edcd58dece6 (DEFAULT_TABLE table=test-table [id=2e15ecbf86924062a71b40c15dda045d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:40:05.461308 8805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e45134135987459e9da32edcd58dece6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:40:05.476289 8957 tablet_bootstrap.cc:492] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: Bootstrap starting.
I20260810 14:40:05.478693 8958 tablet_bootstrap.cc:492] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: Bootstrap starting.
I20260810 14:40:05.482609 8957 tablet_bootstrap.cc:654] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: Neither blocks nor log segments found. Creating new log.
I20260810 14:40:05.483922 8958 tablet_bootstrap.cc:654] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: Neither blocks nor log segments found. Creating new log.
I20260810 14:40:05.484220 8957 log.cc:824] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:40:05.485139 8960 tablet_bootstrap.cc:492] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: Bootstrap starting.
I20260810 14:40:05.485489 8958 log.cc:824] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:40:05.488519 8960 tablet_bootstrap.cc:654] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: Neither blocks nor log segments found. Creating new log.
I20260810 14:40:05.489315 8957 tablet_bootstrap.cc:492] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: No bootstrap required, opened a new log
I20260810 14:40:05.489313 8958 tablet_bootstrap.cc:492] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: No bootstrap required, opened a new log
I20260810 14:40:05.489449 8960 log.cc:824] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: Log is configured to *not* fsync() on all Append() calls
I20260810 14:40:05.489590 8958 ts_tablet_manager.cc:1397] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260810 14:40:05.489590 8957 ts_tablet_manager.cc:1397] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.009s
I20260810 14:40:05.496527 8960 tablet_bootstrap.cc:492] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: No bootstrap required, opened a new log
I20260810 14:40:05.496855 8960 ts_tablet_manager.cc:1397] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.005s
I20260810 14:40:05.501988 8957 raft_consensus.cc:348] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051641f4857b404a98d7dcce5376b57a" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34495 } } peers { permanent_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44955 } } peers { permanent_uuid: "dace796d542945c7975fd1861594f2c6" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42791 } }
I20260810 14:40:05.501988 8958 raft_consensus.cc:348] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051641f4857b404a98d7dcce5376b57a" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34495 } } peers { permanent_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44955 } } peers { permanent_uuid: "dace796d542945c7975fd1861594f2c6" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42791 } }
I20260810 14:40:05.502499 8958 raft_consensus.cc:374] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:40:05.502499 8957 raft_consensus.cc:374] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:40:05.502655 8957 raft_consensus.cc:729] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051641f4857b404a98d7dcce5376b57a, State: Initialized, Role: FOLLOWER
I20260810 14:40:05.502701 8958 raft_consensus.cc:729] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dace796d542945c7975fd1861594f2c6, State: Initialized, Role: FOLLOWER
I20260810 14:40:05.503722 8957 consensus_queue.cc:260] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051641f4857b404a98d7dcce5376b57a" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34495 } } peers { permanent_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44955 } } peers { permanent_uuid: "dace796d542945c7975fd1861594f2c6" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42791 } }
I20260810 14:40:05.503722 8958 consensus_queue.cc:260] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051641f4857b404a98d7dcce5376b57a" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34495 } } peers { permanent_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44955 } } peers { permanent_uuid: "dace796d542945c7975fd1861594f2c6" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42791 } }
I20260810 14:40:05.505074 8680 heartbeater.cc:503] Master 127.21.55.126:36895 was elected leader, sending a full tablet report...
I20260810 14:40:05.506106 8957 ts_tablet_manager.cc:1428] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260810 14:40:05.506451 8960 raft_consensus.cc:348] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051641f4857b404a98d7dcce5376b57a" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34495 } } peers { permanent_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44955 } } peers { permanent_uuid: "dace796d542945c7975fd1861594f2c6" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42791 } }
I20260810 14:40:05.506984 8960 raft_consensus.cc:374] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:40:05.507128 8960 raft_consensus.cc:729] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cee78e0796604ad6bd98a67c6c4ce4f1, State: Initialized, Role: FOLLOWER
I20260810 14:40:05.507893 8958 ts_tablet_manager.cc:1428] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: Time spent starting tablet: real 0.018s user 0.010s sys 0.009s
I20260810 14:40:05.507812 8960 consensus_queue.cc:260] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051641f4857b404a98d7dcce5376b57a" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34495 } } peers { permanent_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44955 } } peers { permanent_uuid: "dace796d542945c7975fd1861594f2c6" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42791 } }
I20260810 14:40:05.509068 8903 heartbeater.cc:503] Master 127.21.55.126:36895 was elected leader, sending a full tablet report...
I20260810 14:40:05.509799 8960 ts_tablet_manager.cc:1428] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260810 14:40:05.555361 8907 tablet.cc:2369] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:40:05.564365 8970 raft_consensus.cc:482] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:40:05.564749 8970 raft_consensus.cc:504] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051641f4857b404a98d7dcce5376b57a" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34495 } } peers { permanent_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44955 } } peers { permanent_uuid: "dace796d542945c7975fd1861594f2c6" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42791 } }
I20260810 14:40:05.566851 8970 leader_election.cc:288] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051641f4857b404a98d7dcce5376b57a (127.21.55.66:34495), cee78e0796604ad6bd98a67c6c4ce4f1 (127.21.55.67:44955)
I20260810 14:40:05.575793 8947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.567915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:52603 (local address 127.21.55.67:44955)
0810 14:40:05.568242 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:05.568251 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:05.569209 (+ 958us) server_negotiation.cc:407] Connection header received
0810 14:40:05.569496 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:05.569504 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:05.569586 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:05.569654 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:40:05.570562 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.571463 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.573472 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.574668 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.574829 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.574909 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:05.575392 (+ 483us) server_negotiation.cc:299] Negotiation successful
0810 14:40:05.575536 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 14:40:05.576066 8933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.567371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:46993 (local address 127.21.55.66:34495)
0810 14:40:05.567674 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:05.567684 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:05.569012 (+ 1328us) server_negotiation.cc:407] Connection header received
0810 14:40:05.569242 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:05.569246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:05.569305 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:05.569409 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:40:05.569957 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.570958 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.573796 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.575043 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.575216 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.575293 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:05.575694 (+ 401us) server_negotiation.cc:299] Negotiation successful
0810 14:40:05.575820 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260810 14:40:05.576656 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.567782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:44955 (local address 127.21.55.65:52603)
0810 14:40:05.568939 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:05.568960 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:40:05.569333 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:40:05.569899 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:40:05.569906 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:40:05.569933 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:40:05.570366 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.570382 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.571596 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.571602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.573304 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.573317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.574912 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.574921 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.575144 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.575196 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 14:40:05.576123 (+ 927us) client_negotiation.cc:240] Negotiation successful
0810 14:40:05.576257 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"mutex_wait_us":1,"thread_start_us":585,"threads_started":1}
I20260810 14:40:05.578263 8836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e45134135987459e9da32edcd58dece6" candidate_uuid: "dace796d542945c7975fd1861594f2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" is_pre_election: true
I20260810 14:40:05.579110 8836 raft_consensus.cc:2454] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dace796d542945c7975fd1861594f2c6 in term 0.
I20260810 14:40:05.580027 8258 leader_election.cc:302] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cee78e0796604ad6bd98a67c6c4ce4f1, dace796d542945c7975fd1861594f2c6; no voters:
I20260810 14:40:05.580586 8970 raft_consensus.cc:2781] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:40:05.580734 8970 raft_consensus.cc:482] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:40:05.580830 8970 raft_consensus.cc:3037] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:40:05.585140 8970 raft_consensus.cc:504] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051641f4857b404a98d7dcce5376b57a" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34495 } } peers { permanent_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44955 } } peers { permanent_uuid: "dace796d542945c7975fd1861594f2c6" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42791 } }
I20260810 14:40:05.586000 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.567080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:34495 (local address 127.21.55.65:46993)
0810 14:40:05.568757 (+ 1677us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:05.568783 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:40:05.569091 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:40:05.569435 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:40:05.569440 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:40:05.569466 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:40:05.569802 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.569820 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.571084 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.571093 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.573616 (+ 2523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:05.573633 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.575262 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.575268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:05.575506 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.575544 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 14:40:05.576377 (+ 833us) client_negotiation.cc:240] Negotiation successful
0810 14:40:05.576546 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1529,"thread_start_us":1488,"threads_started":1}
I20260810 14:40:05.586520 8836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e45134135987459e9da32edcd58dece6" candidate_uuid: "dace796d542945c7975fd1861594f2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1"
I20260810 14:40:05.586791 8836 raft_consensus.cc:3037] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:40:05.587499 8612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e45134135987459e9da32edcd58dece6" candidate_uuid: "dace796d542945c7975fd1861594f2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051641f4857b404a98d7dcce5376b57a" is_pre_election: true
I20260810 14:40:05.587486 8611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e45134135987459e9da32edcd58dece6" candidate_uuid: "dace796d542945c7975fd1861594f2c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051641f4857b404a98d7dcce5376b57a"
I20260810 14:40:05.588258 8611 raft_consensus.cc:3037] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:40:05.588635 8970 leader_election.cc:288] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [CANDIDATE]: Term 1 election: Requested vote from peers 051641f4857b404a98d7dcce5376b57a (127.21.55.66:34495), cee78e0796604ad6bd98a67c6c4ce4f1 (127.21.55.67:44955)
I20260810 14:40:05.591341 8836 raft_consensus.cc:2454] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dace796d542945c7975fd1861594f2c6 in term 1.
I20260810 14:40:05.591894 8258 leader_election.cc:302] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cee78e0796604ad6bd98a67c6c4ce4f1, dace796d542945c7975fd1861594f2c6; no voters:
I20260810 14:40:05.592260 8970 raft_consensus.cc:2781] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:40:05.593048 8970 raft_consensus.cc:686] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 1 LEADER]: Becoming Leader. State: Replica: dace796d542945c7975fd1861594f2c6, State: Running, Role: LEADER
I20260810 14:40:05.593333 8611 raft_consensus.cc:2454] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dace796d542945c7975fd1861594f2c6 in term 1.
I20260810 14:40:05.593843 8970 consensus_queue.cc:237] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051641f4857b404a98d7dcce5376b57a" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34495 } } peers { permanent_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44955 } } peers { permanent_uuid: "dace796d542945c7975fd1861594f2c6" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42791 } }
I20260810 14:40:05.602337 8057 catalog_manager.cc:5495] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to dace796d542945c7975fd1861594f2c6 (127.21.55.65). New cstate: current_term: 1 leader_uuid: "dace796d542945c7975fd1861594f2c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "051641f4857b404a98d7dcce5376b57a" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dace796d542945c7975fd1861594f2c6" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42791 } health_report { overall_health: HEALTHY } } }
I20260810 14:40:05.672242 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.661605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38248 (local address 127.21.55.65:42791)
0810 14:40:05.661983 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:05.661992 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:05.662022 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:40:05.662258 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:05.662266 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:05.662447 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:05.662685 (+ 238us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:40:05.663358 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.664707 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.667797 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.668732 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.668903 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.669802 (+ 899us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:40:05.670742 (+ 940us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:40:05.670905 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:05.671018 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 14:40:05.671118 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260810 14:40:05.685575 8836 raft_consensus.cc:1261] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1 [term 1 FOLLOWER]: Refusing update from remote peer dace796d542945c7975fd1861594f2c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:40:05.686743 8970 consensus_queue.cc:1035] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cee78e0796604ad6bd98a67c6c4ce4f1" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 44955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 14:40:05.688583 8440 tablet.cc:2369] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:40:05.690343 8611 raft_consensus.cc:1261] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a [term 1 FOLLOWER]: Refusing update from remote peer dace796d542945c7975fd1861594f2c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:40:05.691316 8970 consensus_queue.cc:1035] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051641f4857b404a98d7dcce5376b57a" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 14:40:05.714159 8683 tablet.cc:2369] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:40:05.721679 8988 mvcc.cc:205] Tried to move back new op lower bound from 7316983012073893888 to 7316983011720744960. Current Snapshot: MvccSnapshot[applied={T|T < 7316983012073893888}]
I20260810 14:40:05.737721 8057 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:40:05.738196 8057 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:40:05.745522 8057 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4a2734547ef648f59e016a2fe27236b4: Sending DeleteTablet for 3 replicas of tablet e45134135987459e9da32edcd58dece6
I20260810 14:40:05.746941 8805 tablet_service.cc:1514] Processing DeleteTablet for tablet e45134135987459e9da32edcd58dece6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:40:05 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:48080
I20260810 14:40:05.746985 8577 tablet_service.cc:1514] Processing DeleteTablet for tablet e45134135987459e9da32edcd58dece6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:40:05 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:52264
I20260810 14:40:05.748607 8998 tablet_replica.cc:330] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: stopping tablet replica
I20260810 14:40:05.749313 8998 raft_consensus.cc:2229] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:40:05.749357 8324 tablet_service.cc:1514] Processing DeleteTablet for tablet e45134135987459e9da32edcd58dece6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:40:05 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:38240
I20260810 14:40:05.750123 8998 raft_consensus.cc:2258] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:40:05.754868 8998 ts_tablet_manager.cc:1905] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:40:05.762410 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 8211
I20260810 14:40:05.762509 8996 tablet_replica.cc:330] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: stopping tablet replica
I20260810 14:40:05.763295 8996 raft_consensus.cc:2229] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:40:05.764182 8996 raft_consensus.cc:2258] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:40:05.764477 8998 ts_tablet_manager.cc:1918] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:40:05.764638 8998 log.cc:1196] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/e45134135987459e9da32edcd58dece6
I20260810 14:40:05.765894 8998 ts_tablet_manager.cc:1939] T e45134135987459e9da32edcd58dece6 P cee78e0796604ad6bd98a67c6c4ce4f1: Deleting consensus metadata
I20260810 14:40:05.766450 8996 ts_tablet_manager.cc:1905] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:40:05.768249 8036 catalog_manager.cc:4841] TS cee78e0796604ad6bd98a67c6c4ce4f1 (127.21.55.67:44955): tablet e45134135987459e9da32edcd58dece6 (table test-table [id=2e15ecbf86924062a71b40c15dda045d]) successfully deleted
I20260810 14:40:05.774767 8996 ts_tablet_manager.cc:1918] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:40:05.775099 8996 log.cc:1196] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/e45134135987459e9da32edcd58dece6
I20260810 14:40:05.775789 8996 ts_tablet_manager.cc:1939] T e45134135987459e9da32edcd58dece6 P 051641f4857b404a98d7dcce5376b57a: Deleting consensus metadata
I20260810 14:40:05.778368 8031 catalog_manager.cc:4841] TS 051641f4857b404a98d7dcce5376b57a (127.21.55.66:34495): tablet e45134135987459e9da32edcd58dece6 (table test-table [id=2e15ecbf86924062a71b40c15dda045d]) successfully deleted
I20260810 14:40:05.783329 8999 tablet_replica.cc:330] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: stopping tablet replica
I20260810 14:40:05.784108 8999 raft_consensus.cc:2229] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:40:05.785601 8999 raft_consensus.cc:2258] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 10 14:40:05 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:40:05.788741 8999 ts_tablet_manager.cc:1905] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:40:05.801173 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.767573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38262 (local address 127.21.55.65:42791)
0810 14:40:05.767772 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:05.767780 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:05.769257 (+ 1477us) server_negotiation.cc:407] Connection header received
0810 14:40:05.769587 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:05.769594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:05.769733 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:05.769830 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:40:05.772306 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.773700 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.776245 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.777552 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.777730 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.787892 (+ 10162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:40:05.787920 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:40:05.787940 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:40:05.787997 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:40:05.794004 (+ 6007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:05.795250 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:05.795260 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:05.795264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:05.795571 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:05.796181 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:05.796186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:05.796190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:05.796739 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:40:05.799559 (+ 2820us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:05.799986 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 14:40:05.800434 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 14:40:05.802153 8999 ts_tablet_manager.cc:1918] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:40:05.802373 8999 log.cc:1196] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/e45134135987459e9da32edcd58dece6
I20260810 14:40:05.803158 8999 ts_tablet_manager.cc:1939] T e45134135987459e9da32edcd58dece6 P dace796d542945c7975fd1861594f2c6: Deleting consensus metadata
I20260810 14:40:05.803354 8430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:40:05.913494 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 8211
I20260810 14:40:05.940042 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 8453
W20260810 14:40:05.940023 8035 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.55.65:42791 (error 108)
I20260810 14:40:05.942008 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.941501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42791 (local address 127.0.0.1:38272)
0810 14:40:05.941692 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:05.941819 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 14:40:05.942286 8035 catalog_manager.cc:4584] TS dace796d542945c7975fd1861594f2c6 (127.21.55.65:42791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e45134135987459e9da32edcd58dece6: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42791: connect: Connection refused (error 111)
Aug 10 14:40:05 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:40:05.959367 8933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.940933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52272 (local address 127.21.55.66:34495)
0810 14:40:05.941131 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:05.941136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:05.941258 (+ 122us) server_negotiation.cc:407] Connection header received
0810 14:40:05.941461 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:05.941468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:05.941572 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:05.941703 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:40:05.943233 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.943977 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.945701 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:05.946115 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:05.946219 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:05.951267 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:40:05.951282 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:40:05.951298 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:40:05.951359 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:40:05.954818 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:05.956157 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:05.956166 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:05.956168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:05.956335 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:05.957063 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:05.957069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:05.957072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:05.957712 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:40:05.958046 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:05.958477 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 14:40:05.958916 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:40:05.960379 8672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:40:05.996191 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.995640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42791 (local address 127.0.0.1:38280)
0810 14:40:05.995894 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:05.996009 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 14:40:05.997328 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:05.996942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42791 (local address 127.0.0.1:38288)
0810 14:40:05.997101 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:05.997184 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 14:40:06.054618 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 8453
I20260810 14:40:06.058959 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:06.058054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42791 (local address 127.0.0.1:38294)
0810 14:40:06.058462 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:06.058654 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222}
I20260810 14:40:06.061064 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:06.060370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42791 (local address 127.0.0.1:38304)
0810 14:40:06.060683 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:06.060810 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260810 14:40:06.083122 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 8693
Aug 10 14:40:06 dist-test-slave-h6n0 krb5kdc[7283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:40:06.099944 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:06.084059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48090 (local address 127.21.55.67:44955)
0810 14:40:06.084582 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:06.084587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:06.084606 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:40:06.084667 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:06.084671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:06.084745 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:06.084856 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:40:06.086160 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:06.087218 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:06.088739 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:06.089341 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:06.089443 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:06.094035 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:40:06.094050 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:40:06.094069 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:40:06.094101 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:40:06.097030 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:06.097679 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:06.097686 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:06.097688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:06.097842 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:06.098193 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:06.098196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:06.098198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:06.098673 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:40:06.098950 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:06.099314 (+ 364us) server_negotiation.cc:299] Negotiation successful
0810 14:40:06.099510 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":296,"threads_started":1}
I20260810 14:40:06.100728 8897 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:40:06.142022 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:06.141449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42791 (local address 127.0.0.1:38314)
0810 14:40:06.141685 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:06.141796 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 14:40:06.143169 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:06.142756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42791 (local address 127.0.0.1:38320)
0810 14:40:06.142966 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:06.143048 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260810 14:40:06.191425 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 8693
I20260810 14:40:06.213408 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 7995
I20260810 14:40:06.215101 8121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:40:06.337996 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:06.337188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42791 (local address 127.0.0.1:38330)
0810 14:40:06.337421 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:06.337602 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 14:40:06.338732 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 7995
I20260810 14:40:06.377290 21725 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 14:40:06.724426 21725 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 14:40:06.724725 21725 mini_ranger.cc:66] Stopping Ranger...
I20260810 14:40:07.082005 21725 mini_ranger.cc:68] Stopped Ranger
2026-08-10 14:40:07.082 UTC [7332] LOG: received smart shutdown request
2026-08-10 14:40:07.086 UTC [7332] LOG: background worker "logical replication launcher" (PID 7343) exited with exit code 1
2026-08-10 14:40:07.087 UTC [7335] LOG: shutting down
2026-08-10 14:40:07.273 UTC [7332] LOG: database system is shut down
2026-08-10T14:40:07Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54914 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:40:07 dist-test-slave-h6n0 krb5kdc[9029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:40:07 dist-test-slave-h6n0 krb5kdc[9029](info): set up 2 sockets
Aug 10 14:40:07 dist-test-slave-h6n0 krb5kdc[9029](info): commencing operation
krb5kdc: starting...
W20260810 14:40:08.875554 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.413s user 0.007s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:40:09 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:40:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:40:09Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.55.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.55.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/rangeradmin_127.21.55.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/rangeradmin_127.21.55.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.55.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.55.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/rangerlookup_127.21.55.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/rangerlookup_127.21.55.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.55.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.55.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/HTTP_127.21.55.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/HTTP_127.21.55.84@KRBTEST.COM.keytab.
I20260810 14:40:09.287524 21725 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwtLR1L/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/postgres -l logfile start
2026-08-10 14:40:12.090 UTC [9084] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 14:40:12.090 UTC [9084] LOG: listening on IPv4 address "127.21.55.84", port 34167
2026-08-10 14:40:12.095 UTC [9084] LOG: listening on Unix socket "/tmp/.s.PGSQL.34167"
2026-08-10 14:40:12.117 UTC [9089] LOG: database system was shut down at 2026-08-10 14:40:10 UTC
2026-08-10 14:40:12.120 UTC [9084] LOG: database system is ready to accept connections
I20260810 14:40:13.439288 21725 mini_postgres.cc:96] Postgres bound to 34167
2026-08-10 14:40:13.464 UTC [9103] FATAL: database "slave" does not exist
127.21.55.84:34167 - accepting connections
I20260810 14:40:13.464857 21725 mini_ranger.cc:161] Starting Ranger...
I20260810 14:40:13.496701 21725 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 14:40:14.058995 21725 mini_ranger.cc:87] Created ranger Postgres database
I20260810 14:40:14.059137 21725 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-admin
2026-08-10 14:40:14,393 [I] DB FLAVOR :POSTGRES
2026-08-10 14:40:14,394 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:40:14,394 [I] Checking connection..
2026-08-10 14:40:14,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 14:40:14,686 [I] Checking connection passed.
2026-08-10 14:40:14,687 [I] --------- Verifying version history table ---------
2026-08-10 14:40:14,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:40:14,965 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:40:14,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:40:15,232 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:40:15,232 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 14:40:15,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 14:40:15.509 UTC [9217] WARNING: there is no transaction in progress
2026-08-10 14:40:15,518 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 14:40:15,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:40:15,784 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 14:40:15,785 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:40:15,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 14:40:16,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 14:40:16,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h6n0.c.gcp-upstream.internal','N') ;"
2026-08-10 14:40:16,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:40:16,874 [I] Table x_portal_user does not exist in database ranger
2026-08-10 14:40:16,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:40:17,158 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 14:40:17,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:40:17,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:40:17,729 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 14:40:17,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 14:40:18.833 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:18.853 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:18.873 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:18.890 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:18.909 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:18.928 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:18.986 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:18.994 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:19.009 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:19.020 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:19.030 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:19.046 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:19.057 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:19.066 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:19.075 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:19.084 UTC [9389] WARNING: there is no transaction in progress
2026-08-10 14:40:19,794 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 14:40:19,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 14:40:20,069 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 14:40:20,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:40:20,343 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 14:40:20,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:40:20,631 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 14:40:20,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:40:20,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:40:21,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h6n0.c.gcp-upstream.internal';"
2026-08-10 14:40:21,430 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 14:40:21,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 14:40:21,715 [I] DB_PATCHES have already been applied
I20260810 14:40:21.722170 21725 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwtLR1L/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 14:40:21.722280 21725 mini_ranger.cc:193] Using host: 127.21.55.84
I20260810 14:40:21.742386 21725 mini_ranger.cc:227] Ranger admin URL: http://127.21.55.84:53267
Aug 10, 2026 2:40:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:40:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:40:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 2:40:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 2:40:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 2:40:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 2:40:22 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 14:40:23.212280 21725 mini_ranger.cc:160] Time spent starting Ranger: real 9.747s user 0.010s sys 0.049s
Aug 10, 2026 2:40:45 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 2:40:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 2:40:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 2:40:45 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 2:40:46 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 14:40:46.690735 21725 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.55.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.55.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/rangerkms_127.21.55.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.55.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/rangerkms_127.21.55.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.55.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.55.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.55.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/HTTP_127.21.55.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.55.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/HTTP_127.21.55.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 14:40:46 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372846, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 14:40:46.897495 21725 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 14:40:46.923822 21725 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 14:40:47.850917 21725 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 14:40:47.851034 21725 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-kms
I20260810 14:40:47.851073 21725 mini_ranger_kms.cc:227] Using postgres at 127.21.55.84:34167
2026-08-10 14:40:48,264 [I] DB FLAVOR :POSTGRES
2026-08-10 14:40:48,264 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:40:48,264 [I] Checking connection
2026-08-10 14:40:48,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 14:40:48,590 [I] connection success
2026-08-10 14:40:48,590 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 14:40:48,590 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 14:40:48,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 14:40:48,910 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 14:40:48,911 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:40:48,911 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 14:40:48,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.55.84:34167/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 14:40:49,210 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 14:40:49.776928 21725 mini_ranger_kms.cc:310] Created kms service
I20260810 14:40:50.023056 21725 mini_ranger_kms.cc:326] Created kudu user
I20260810 14:40:50.081180 21725 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 14:40:50 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372846, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:40:50.537456 21725 mini_ranger_kms.cc:384] Added ranger policy
I20260810 14:40:50.537813 21725 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwtLR1L/build/asan/bin/postgresql.jar:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwtLR1L/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 14:40:50.537864 21725 mini_ranger_kms.cc:240] Using host: 127.21.55.84
I20260810 14:40:50.562028 21725 mini_ranger_kms.cc:276] Ranger KMS PID: 9693
I20260810 14:40:50.562165 21725 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.55.84:49927
Aug 10, 2026 2:40:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:40:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:40:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 2:40:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 2:40:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 2:40:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 14:40:52 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786372852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.55.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 14:40:53 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.55.84@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:40:53.609760 21725 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.712s user 0.091s sys 0.348s
I20260810 14:40:53.610190 21725 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 14:40:53.610394 21725 mini_ranger_kms.cc:401] 127.21.55.84:49927/kms/v1/keys
Aug 10 14:40:53 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.55.84@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
Aug 10 14:40:56 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:40:56.254417 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:33111
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:33111
--ranger_config_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:49927/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:40:56.622128 9775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:40:56.622417 9775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:40:56.622598 9775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:40:56.630028 9775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:40:56.630129 9775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:40:56.630231 9775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:40:56.630318 9775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:40:56.630403 9775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:40:56.640281 9775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.55.84:49927/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:33111
--ranger_config_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:33111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:40:56.641578 9775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:40:56.642922 9775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:40:56.655174 9784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:56.655098 9787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:56.655205 9785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:40:56.656324 9775 server_base.cc:1034] running on GCE node
I20260810 14:40:56.657223 9775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:40:56.658815 9775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:40:56.660080 9775 hybrid_clock.cc:648] HybridClock initialized: now 1786372856660031 us; error 36 us; skew 500 ppm
I20260810 14:40:56.660495 9775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:40:56 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:40:56.665722 9775 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:40:56.668033 9775 webserver.cc:460] Webserver started at http://127.21.55.126:38135/ using document root <none> and password file <none>
I20260810 14:40:56.668756 9775 fs_manager.cc:359] Metadata directory not provided
I20260810 14:40:56.668874 9775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:40:56.669162 9775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:40:56 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:40:56.737952 9775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "4e69e54aeea0415d918982183dbfa2d8"
format_stamp: "Formatted at 2026-08-10 14:40:56 on dist-test-slave-h6n0"
server_key: "ee62c2f2205c37e46600dcdf072ada3a"
server_key_iv: "39d5b5541d7a5975061b1f59364b4aad"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:56.739889 9775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "4e69e54aeea0415d918982183dbfa2d8"
format_stamp: "Formatted at 2026-08-10 14:40:56 on dist-test-slave-h6n0"
server_key: "ee62c2f2205c37e46600dcdf072ada3a"
server_key_iv: "39d5b5541d7a5975061b1f59364b4aad"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:56.746304 9775 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 14:40:56.766860 9799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:56.768766 9775 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:40:56.768982 9775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "4e69e54aeea0415d918982183dbfa2d8"
format_stamp: "Formatted at 2026-08-10 14:40:56 on dist-test-slave-h6n0"
server_key: "ee62c2f2205c37e46600dcdf072ada3a"
server_key_iv: "39d5b5541d7a5975061b1f59364b4aad"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:56.769317 9775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:40:56.792618 9775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:40:56.796806 9775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:40:56.797139 9775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:40:56.820135 9775 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:33111
I20260810 14:40:56.820135 9871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:33111 every 8 connection(s)
I20260810 14:40:56.821641 9775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:40:56.825570 9872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:40:56.828215 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 9775
I20260810 14:40:56.828464 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:40:56.829231 21725 external_mini_cluster.cc:1428] Setting key c448e8d80a761dce4c2af6f52d00f010
I20260810 14:40:56.839465 9872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8: Bootstrap starting.
I20260810 14:40:56.843392 9872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8: Neither blocks nor log segments found. Creating new log.
I20260810 14:40:56.844806 9872 log.cc:824] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:40:56 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:40:56.848590 9872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8: No bootstrap required, opened a new log
I20260810 14:40:56.856570 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:56.831397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56724 (local address 127.21.55.126:33111)
0810 14:40:56.832301 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:56.832313 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:56.832356 (+ 43us) server_negotiation.cc:407] Connection header received
0810 14:40:56.834126 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:56.834151 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:56.834634 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:56.835008 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:40:56.836800 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:56.838840 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:56.840719 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:56.841974 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:56.842268 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:56.847511 (+ 5243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:40:56.847526 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:40:56.847547 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:40:56.847596 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:40:56.851168 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:56.852129 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:56.852136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:56.852142 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:56.852323 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:56.852837 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:56.852842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:56.852844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:56.853641 (+ 797us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:40:56.854184 (+ 543us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:56.854761 (+ 577us) server_negotiation.cc:299] Negotiation successful
0810 14:40:56.855436 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":464,"threads_started":1}
I20260810 14:40:56.858484 9872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } }
I20260810 14:40:56.859025 9872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:40:56.859169 9872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e69e54aeea0415d918982183dbfa2d8, State: Initialized, Role: FOLLOWER
I20260810 14:40:56.859860 9872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } }
I20260810 14:40:56.860158 9872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:40:56.860353 9872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:40:56.860548 9872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:40:56.863019 9872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } }
I20260810 14:40:56.863564 9872 leader_election.cc:302] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e69e54aeea0415d918982183dbfa2d8; no voters:
I20260810 14:40:56.864290 9872 leader_election.cc:288] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:40:56.864473 9885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:40:56.865626 9885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 1 LEADER]: Becoming Leader. State: Replica: 4e69e54aeea0415d918982183dbfa2d8, State: Running, Role: LEADER
I20260810 14:40:56.866302 9885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } }
I20260810 14:40:56.867290 9872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:40:56.869632 9886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e69e54aeea0415d918982183dbfa2d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } } }
I20260810 14:40:56.870061 9886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [sys.catalog]: This master's current role is: LEADER
I20260810 14:40:56.869753 9887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e69e54aeea0415d918982183dbfa2d8. Latest consensus state: current_term: 1 leader_uuid: "4e69e54aeea0415d918982183dbfa2d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } } }
I20260810 14:40:56.870270 9887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [sys.catalog]: This master's current role is: LEADER
I20260810 14:40:56.874230 9892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:40:56.879683 9892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:40:56.880589 9872 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 14:40:56.903047 9892 catalog_manager.cc:1347] Generated new cluster ID: 0f011ceb512340f69643798857513645
I20260810 14:40:56.903244 9892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:40:56.929019 9892 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:40:56.931185 9892 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:40:56.950433 9892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8: Generated new TSK 0
I20260810 14:40:56.951697 9892 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:40:57.811066 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:49927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33111
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:40:58 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786372858, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 14:40:58.326673 9959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:40:58.327219 9959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:40:58.327450 9959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:40:58.340626 9959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:40:58.340898 9959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:40:58.341166 9959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:40:58.358758 9959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.55.84:49927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:40:58.361388 9959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:40:58.363415 9959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:40:58.386025 9975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:58.393712 9974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:58.394832 9981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:40:58.395483 9959 server_base.cc:1034] running on GCE node
I20260810 14:40:58.396616 9959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:40:58.410759 9959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:40:58.412554 9959 hybrid_clock.cc:648] HybridClock initialized: now 1786372858412410 us; error 139 us; skew 500 ppm
I20260810 14:40:58.413264 9959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:40:58 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372858, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:40:58.421263 9959 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:40:58.432516 9959 webserver.cc:460] Webserver started at http://127.21.55.65:41323/ using document root <none> and password file <none>
I20260810 14:40:58.433728 9959 fs_manager.cc:359] Metadata directory not provided
I20260810 14:40:58.434243 9959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:40:58.434981 9959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:40:58 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372858, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:40:58.462136 9959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "dbe4f06c736c4c15861bfbb51bcad2d3"
format_stamp: "Formatted at 2026-08-10 14:40:58 on dist-test-slave-h6n0"
server_key: "a51f6c99c165bdae503ff96bb3a123ba"
server_key_iv: "c1e2afd485812e6a1a8f68a468f12bfe"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:58.463341 9959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "dbe4f06c736c4c15861bfbb51bcad2d3"
format_stamp: "Formatted at 2026-08-10 14:40:58 on dist-test-slave-h6n0"
server_key: "a51f6c99c165bdae503ff96bb3a123ba"
server_key_iv: "c1e2afd485812e6a1a8f68a468f12bfe"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:58.469931 9959 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260810 14:40:58.495350 9992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:58.499390 9959 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.004s
I20260810 14:40:58.500041 9959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "dbe4f06c736c4c15861bfbb51bcad2d3"
format_stamp: "Formatted at 2026-08-10 14:40:58 on dist-test-slave-h6n0"
server_key: "a51f6c99c165bdae503ff96bb3a123ba"
server_key_iv: "c1e2afd485812e6a1a8f68a468f12bfe"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:58.500779 9959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:40:58.578889 9959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:40:58.586583 9959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:40:58.587112 9959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:40:58.595254 9959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:40:58.603870 9959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:40:58.604125 9959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260810 14:40:58.604373 9959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:40:58.604519 9959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:40:58 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372858, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:40:58.875135 9959 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:38335
I20260810 14:40:58.875187 10173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:38335 every 8 connection(s)
I20260810 14:40:58.877131 9959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:40:58.884068 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 9959
I20260810 14:40:58.884392 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:40:58.885502 21725 external_mini_cluster.cc:1428] Setting key 8f3546b3eb4f97847a15d341998b0990
Aug 10 14:40:58 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372858, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:40:58.924475 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:58.880430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:56631 (local address 127.21.55.126:33111)
0810 14:40:58.891372 (+ 10942us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:58.891380 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:58.891406 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:40:58.892858 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:58.892865 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:58.892948 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:58.893061 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:40:58.895411 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:58.896648 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:58.899215 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:58.899874 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:58.900004 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:58.912452 (+ 12448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:40:58.912464 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:40:58.912473 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:40:58.912550 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:40:58.917303 (+ 4753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:58.919494 (+ 2191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:58.919506 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:58.919513 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:58.919805 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:58.921225 (+ 1420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:58.921235 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:58.921242 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:58.921833 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:40:58.922011 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:58.923423 (+ 1412us) server_negotiation.cc:299] Negotiation successful
0810 14:40:58.923777 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10808,"thread_start_us":10662,"threads_started":1}
I20260810 14:40:58.927702 10179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:58.881605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33111 (local address 127.21.55.65:56631)
0810 14:40:58.889592 (+ 7987us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:58.889664 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 14:40:58.892400 (+ 2736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:40:58.893532 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:40:58.893546 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:40:58.894273 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:40:58.895175 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:58.895201 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:58.896853 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:58.896859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:58.899004 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:58.899022 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:58.900497 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:58.900505 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:58.901267 (+ 762us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:58.907052 (+ 5785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:40:58.907389 (+ 337us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:40:58.912044 (+ 4655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:40:58.917611 (+ 5567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:40:58.917626 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:40:58.917653 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:40:58.919047 (+ 1394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:40:58.920028 (+ 981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:40:58.920036 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:40:58.920042 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:40:58.920854 (+ 812us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:40:58.922163 (+ 1309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:40:58.922178 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:40:58.922859 (+ 681us) client_negotiation.cc:769] Sending connection context
0810 14:40:58.925115 (+ 2256us) client_negotiation.cc:240] Negotiation successful
0810 14:40:58.925898 (+ 783us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7460,"thread_start_us":6921,"threads_started":1}
I20260810 14:40:58.930172 10176 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33111
I20260810 14:40:58.930598 10176 heartbeater.cc:464] Registering TS with master...
I20260810 14:40:58.931701 10176 heartbeater.cc:511] Master 127.21.55.126:33111 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
I20260810 14:40:58.934950 9824 ts_manager.cc:194] Registered new tserver with Master: dbe4f06c736c4c15861bfbb51bcad2d3 (127.21.55.65:38335)
I20260810 14:40:58.937011 9824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:56631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:40:58.988288 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:49927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33111
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:40:59 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372858, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
W20260810 14:40:59.514942 10182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:40:59.515599 10182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:40:59.515803 10182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:40:59.528951 10182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:40:59.529296 10182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:40:59.529614 10182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:40:59.541344 10182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.55.84:49927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:40:59.542825 10182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:40:59.544270 10182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:40:59.563059 10210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:59.563025 10208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:40:59.562948 10207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:40:59.565158 10182 server_base.cc:1034] running on GCE node
I20260810 14:40:59.566161 10182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:40:59.567806 10182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:40:59.571256 10182 hybrid_clock.cc:648] HybridClock initialized: now 1786372859571184 us; error 46 us; skew 500 ppm
I20260810 14:40:59.571712 10182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:40:59 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372859, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:40:59.577217 10182 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:40:59.579926 10182 webserver.cc:460] Webserver started at http://127.21.55.66:40751/ using document root <none> and password file <none>
I20260810 14:40:59.580931 10182 fs_manager.cc:359] Metadata directory not provided
I20260810 14:40:59.581095 10182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:40:59.581512 10182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:40:59 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372859, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:40:59.605690 10182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "954ae02bcec54f67a8ce7ac04e344959"
format_stamp: "Formatted at 2026-08-10 14:40:59 on dist-test-slave-h6n0"
server_key: "cb943078ee68812eb8f633675933cecc"
server_key_iv: "78ec41eaf6a98f7bab3e8452552fc18e"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:59.607700 10182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "954ae02bcec54f67a8ce7ac04e344959"
format_stamp: "Formatted at 2026-08-10 14:40:59 on dist-test-slave-h6n0"
server_key: "cb943078ee68812eb8f633675933cecc"
server_key_iv: "78ec41eaf6a98f7bab3e8452552fc18e"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:59.618695 10182 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.004s
I20260810 14:40:59.652294 10219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:59.655505 10182 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.005s sys 0.000s
I20260810 14:40:59.655984 10182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "954ae02bcec54f67a8ce7ac04e344959"
format_stamp: "Formatted at 2026-08-10 14:40:59 on dist-test-slave-h6n0"
server_key: "cb943078ee68812eb8f633675933cecc"
server_key_iv: "78ec41eaf6a98f7bab3e8452552fc18e"
server_key_version: "kuduclusterkey@0"
I20260810 14:40:59.656608 10182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:40:59.687083 10182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:40:59.696709 10182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:40:59.697336 10182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:40:59.699615 10182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:40:59.702812 10182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:40:59.702963 10182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:59.703258 10182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:40:59.703377 10182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:40:59.853003 10182 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:34405
I20260810 14:40:59.853111 10393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:34405 every 8 connection(s)
I20260810 14:40:59.857429 10182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:40:59.864598 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 10182
I20260810 14:40:59.865113 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:40:59.866859 21725 external_mini_cluster.cc:1428] Setting key e1be1a52c442ab0492dc194d7319e4e6
Aug 10 14:40:59 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372859, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:40:59.939693 10399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:59.867094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:37375 (local address 127.21.55.126:33111)
0810 14:40:59.884017 (+ 16923us) server_negotiation.cc:206] Beginning negotiation
0810 14:40:59.884030 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:40:59.890650 (+ 6620us) server_negotiation.cc:407] Connection header received
0810 14:40:59.893692 (+ 3042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:40:59.893700 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:40:59.893827 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:40:59.893968 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:40:59.899198 (+ 5230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:59.901780 (+ 2582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:59.908753 (+ 6973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:40:59.910181 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:40:59.910422 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:59.921639 (+ 11217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:40:59.921651 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:40:59.921661 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:40:59.921782 (+ 121us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:40:59.931219 (+ 9437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:59.933680 (+ 2461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:59.933690 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:59.933697 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:59.933954 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:40:59.935090 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:40:59.935100 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:40:59.935119 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:40:59.935936 (+ 817us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:40:59.936172 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 14:40:59.938154 (+ 1982us) server_negotiation.cc:299] Negotiation successful
0810 14:40:59.938654 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16540,"thread_start_us":16985,"threads_started":1}
I20260810 14:40:59.943280 10176 heartbeater.cc:503] Master 127.21.55.126:33111 was elected leader, sending a full tablet report...
I20260810 14:40:59.946285 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:40:59.865537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33111 (local address 127.21.55.66:37375)
0810 14:40:59.885563 (+ 20026us) negotiation.cc:107] Waiting for socket to connect
0810 14:40:59.885699 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0810 14:40:59.892860 (+ 7161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:40:59.895253 (+ 2393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:40:59.895283 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:40:59.896854 (+ 1571us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:40:59.898928 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:59.898963 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:59.902270 (+ 3307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:59.902293 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:59.908006 (+ 5713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:40:59.908098 (+ 92us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:40:59.911533 (+ 3435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:40:59.911558 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:40:59.912701 (+ 1143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:40:59.914367 (+ 1666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:40:59.914454 (+ 87us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:40:59.921032 (+ 6578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:40:59.931741 (+ 10709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:40:59.931770 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:40:59.931850 (+ 80us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:40:59.933346 (+ 1496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:40:59.934230 (+ 884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:40:59.934242 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:40:59.934260 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:40:59.934834 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:40:59.936349 (+ 1515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:40:59.936364 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:40:59.937302 (+ 938us) client_negotiation.cc:769] Sending connection context
0810 14:40:59.940947 (+ 3645us) client_negotiation.cc:240] Negotiation successful
0810 14:40:59.942154 (+ 1207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19233,"thread_start_us":24904,"threads_started":1}
I20260810 14:40:59.951620 10395 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33111
I20260810 14:40:59.952335 10395 heartbeater.cc:464] Registering TS with master...
I20260810 14:40:59.954784 10395 heartbeater.cc:511] Master 127.21.55.126:33111 requested a full tablet report, sending...
I20260810 14:40:59.959671 9824 ts_manager.cc:194] Registered new tserver with Master: 954ae02bcec54f67a8ce7ac04e344959 (127.21.55.66:34405)
I20260810 14:40:59.963078 9824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:37375
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
W20260810 14:41:00.038157 21725 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.55.67: real 0.159s user 0.000s sys 0.043s
I20260810 14:41:00.039855 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:49927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33111
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:00.659771 10404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:00.660310 10404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:00.660437 10404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:00.673549 10404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:00.673806 10404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:00.674124 10404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:41:00.692135 10404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.55.84:49927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:33111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:00.694290 10404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:00.696367 10404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:00.720361 10416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:00.720361 10417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:00.720265 10423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:00.722808 10404 server_base.cc:1034] running on GCE node
I20260810 14:41:00.724103 10404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:00.726110 10404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:00.729651 10404 hybrid_clock.cc:648] HybridClock initialized: now 1786372860729566 us; error 90 us; skew 500 ppm
I20260810 14:41:00.730371 10404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:00 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372860, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:00.736608 10404 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:41:00.739526 10404 webserver.cc:460] Webserver started at http://127.21.55.67:43105/ using document root <none> and password file <none>
I20260810 14:41:00.740887 10404 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:00.741165 10404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:00.742609 10404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:41:00 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372860, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:41:00.773150 10404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "aa543c17462f41e2bda4fdbf98dea5b4"
format_stamp: "Formatted at 2026-08-10 14:41:00 on dist-test-slave-h6n0"
server_key: "a5e327a064d64b24aeee0b8ca0e6c629"
server_key_iv: "7770f408d1550bba8d2c373236a567c4"
server_key_version: "kuduclusterkey@0"
I20260810 14:41:00.775105 10404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "aa543c17462f41e2bda4fdbf98dea5b4"
format_stamp: "Formatted at 2026-08-10 14:41:00 on dist-test-slave-h6n0"
server_key: "a5e327a064d64b24aeee0b8ca0e6c629"
server_key_iv: "7770f408d1550bba8d2c373236a567c4"
server_key_version: "kuduclusterkey@0"
I20260810 14:41:00.786357 10404 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.005s
I20260810 14:41:00.804385 10431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:00.806625 10404 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260810 14:41:00.806867 10404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "aa543c17462f41e2bda4fdbf98dea5b4"
format_stamp: "Formatted at 2026-08-10 14:41:00 on dist-test-slave-h6n0"
server_key: "a5e327a064d64b24aeee0b8ca0e6c629"
server_key_iv: "7770f408d1550bba8d2c373236a567c4"
server_key_version: "kuduclusterkey@0"
I20260810 14:41:00.807108 10404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:00.853641 10404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:00.858111 10404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:00.858480 10404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:00.859635 10404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:00.861181 10404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:00.861299 10404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:00.861454 10404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:00.861548 10404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:00.906878 10404 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:42849
I20260810 14:41:00.906934 10619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:42849 every 8 connection(s)
I20260810 14:41:00.908396 10404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:41:00.909652 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 10404
I20260810 14:41:00.909860 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:41:00.910513 21725 external_mini_cluster.cc:1428] Setting key 8fc90d8a4efc610e84c421a68accec03
Aug 10 14:41:00 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372860, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:00.931864 10622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:00.911482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:57151 (local address 127.21.55.126:33111)
0810 14:41:00.912098 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:00.912104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:00.913038 (+ 934us) server_negotiation.cc:407] Connection header received
0810 14:41:00.915032 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:00.915036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:00.915133 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:00.915243 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:00.916774 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:00.917563 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:00.919419 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:00.919829 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:00.919944 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:00.924001 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:00.924009 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:00.924014 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:00.924078 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:00.928492 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:00.929398 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:00.929408 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:00.929410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:00.929552 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:00.930015 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:00.930019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:00.930021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:00.930316 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:00.930444 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:00.931396 (+ 952us) server_negotiation.cc:299] Negotiation successful
0810 14:41:00.931597 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":290,"threads_started":1}
I20260810 14:41:00.932981 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:00.912056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33111 (local address 127.21.55.67:57151)
0810 14:41:00.912883 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:00.912930 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:00.914752 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:00.915517 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:00.915525 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:00.915981 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:00.916568 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:00.916585 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:00.917736 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:00.917740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:00.919264 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:00.919278 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:00.920106 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:00.920111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:00.920319 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:00.921075 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:00.921101 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:00.923576 (+ 2475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:00.928669 (+ 5093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:00.928677 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:00.928693 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:00.929170 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:00.929671 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:00.929674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:00.929676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:00.929870 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:00.930512 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:00.930519 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:00.930957 (+ 438us) client_negotiation.cc:769] Sending connection context
0810 14:41:00.931245 (+ 288us) client_negotiation.cc:240] Negotiation successful
0810 14:41:00.931685 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":362,"threads_started":1}
I20260810 14:41:00.935134 10620 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33111
I20260810 14:41:00.935671 10620 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:00.936714 10620 heartbeater.cc:511] Master 127.21.55.126:33111 requested a full tablet report, sending...
I20260810 14:41:00.938720 9824 ts_manager.cc:194] Registered new tserver with Master: aa543c17462f41e2bda4fdbf98dea5b4 (127.21.55.67:42849)
I20260810 14:41:00.939905 9824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:57151
I20260810 14:41:00.941708 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:41:00.943645 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 9959
Aug 10 14:41:00 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:41:00.966463 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:00.944902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52872 (local address 127.21.55.65:38335)
0810 14:41:00.945691 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:00.945698 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:00.945724 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:41:00.945785 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:00.945792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:00.946030 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:00.946164 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:00.948613 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:00.949853 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:00.952071 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:00.952967 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:00.953103 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:00.959340 (+ 6237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:00.959356 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:00.959372 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:00.959414 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:00.962462 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:00.963445 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:00.963452 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:00.963454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:00.963632 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:00.964276 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:00.964283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:00.964285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:00.964854 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:00.965330 (+ 476us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:00.965543 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 14:41:00.965894 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":450,"threads_started":1}
I20260810 14:41:00.968509 10167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:00.970824 10395 heartbeater.cc:503] Master 127.21.55.126:33111 was elected leader, sending a full tablet report...
I20260810 14:41:01.056792 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 9959
I20260810 14:41:01.084864 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 10182
Aug 10 14:41:01 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:41:01.117939 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:01.086636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44308 (local address 127.21.55.66:34405)
0810 14:41:01.087519 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:01.087527 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:01.087557 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:41:01.087622 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:01.087629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:01.087865 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:01.088014 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:01.089718 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:01.091038 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:01.093140 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:01.093811 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:01.093934 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:01.106166 (+ 12232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:01.106184 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:01.106201 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:01.106267 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:01.113691 (+ 7424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:01.114801 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:01.114809 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:01.114811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:01.115019 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:01.115601 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:01.115607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:01.115610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:01.116230 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:01.116574 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:01.116978 (+ 404us) server_negotiation.cc:299] Negotiation successful
0810 14:41:01.117466 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":558,"threads_started":1}
I20260810 14:41:01.120033 10387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:01.208280 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 10182
I20260810 14:41:01.236563 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 10404
Aug 10 14:41:01 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:41:01.259791 10633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:01.238015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37836 (local address 127.21.55.67:42849)
0810 14:41:01.238903 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:01.238910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:01.238945 (+ 35us) server_negotiation.cc:407] Connection header received
0810 14:41:01.239011 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:01.239017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:01.239227 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:01.239364 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:01.240733 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:01.241929 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:01.243304 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:01.244316 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:01.244432 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:01.250382 (+ 5950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:01.250396 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:01.250413 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:01.250455 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:01.254977 (+ 4522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:01.256069 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:01.256182 (+ 113us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:01.256185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:01.256359 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:01.257058 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:01.257063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:01.257065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:01.257481 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:01.257762 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:01.258986 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0810 14:41:01.259324 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":429,"threads_started":1}
I20260810 14:41:01.261467 10608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:01.351002 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 10404
I20260810 14:41:01.377375 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 9775
I20260810 14:41:01.378783 9863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:01.490263 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 9775
I20260810 14:41:01.520457 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:33111
--webserver_interface=127.21.55.126
--webserver_port=38135
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:33111
--ranger_config_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:49927/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.55.84:49927/kms with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:01.882884 10640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:01.883189 10640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:01.883347 10640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:01.890921 10640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:41:01.891039 10640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:01.891089 10640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:01.891140 10640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:41:01.891214 10640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:41:01.902029 10640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.55.84:49927/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:33111
--ranger_config_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:33111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=38135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:01.903488 10640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:01.904882 10640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:01.919077 10651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:01.919085 10652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:01.919183 10654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:01.920181 10640 server_base.cc:1034] running on GCE node
I20260810 14:41:01.920949 10640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:01.922668 10640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:01.923954 10640 hybrid_clock.cc:648] HybridClock initialized: now 1786372861923919 us; error 52 us; skew 500 ppm
I20260810 14:41:01.924378 10640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:01 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372861, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:01.928172 10640 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:41:01.929893 10640 webserver.cc:460] Webserver started at http://127.21.55.126:38135/ using document root <none> and password file <none>
I20260810 14:41:01.930620 10640 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:01.930784 10640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:41:01 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372861, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:41:01.983960 10640 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 14:41:01.987200 10667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:01.988528 10640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:41:01.988751 10640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "4e69e54aeea0415d918982183dbfa2d8"
format_stamp: "Formatted at 2026-08-10 14:40:56 on dist-test-slave-h6n0"
server_key: "ee62c2f2205c37e46600dcdf072ada3a"
server_key_iv: "39d5b5541d7a5975061b1f59364b4aad"
server_key_version: "kuduclusterkey@0"
I20260810 14:41:01.989794 10640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:02.026777 10640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:02.030896 10640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:02.031193 10640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:02.052105 10640 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:33111
I20260810 14:41:02.052392 10743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:33111 every 8 connection(s)
I20260810 14:41:02.053663 10640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:41:02.060662 10748 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:41:02.061403 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 10640
I20260810 14:41:02.062857 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:49927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:38335
--local_ip_for_outbound_sockets=127.21.55.65
--tserver_master_addrs=127.21.55.126:33111
--webserver_port=41323
--webserver_interface=127.21.55.65
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.55.84:49927/kms with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:41:02.064080 10748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8: Bootstrap starting.
I20260810 14:41:02.086689 10748 log.cc:824] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:02.099656 10748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:41:02.100332 10748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8: Bootstrap complete.
I20260810 14:41:02.115480 10748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } }
I20260810 14:41:02.116912 10748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e69e54aeea0415d918982183dbfa2d8, State: Initialized, Role: FOLLOWER
I20260810 14:41:02.117784 10748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } }
I20260810 14:41:02.118093 10748 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:41:02.118225 10748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:41:02.118423 10748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:41:02.124585 10748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } }
I20260810 14:41:02.125268 10748 leader_election.cc:302] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e69e54aeea0415d918982183dbfa2d8; no voters:
I20260810 14:41:02.125990 10748 leader_election.cc:288] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:41:02.126327 10754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:41:02.127568 10754 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [term 2 LEADER]: Becoming Leader. State: Replica: 4e69e54aeea0415d918982183dbfa2d8, State: Running, Role: LEADER
I20260810 14:41:02.128203 10754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } }
I20260810 14:41:02.128451 10748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:41:02.130764 10757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e69e54aeea0415d918982183dbfa2d8. Latest consensus state: current_term: 2 leader_uuid: "4e69e54aeea0415d918982183dbfa2d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } } }
I20260810 14:41:02.131162 10757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:02.131706 10756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4e69e54aeea0415d918982183dbfa2d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e69e54aeea0415d918982183dbfa2d8" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 33111 } } }
I20260810 14:41:02.132012 10756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:02.139653 10764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:41:02.147853 10764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:41:02.149879 10764 catalog_manager.cc:1259] Loaded cluster ID: 0f011ceb512340f69643798857513645
I20260810 14:41:02.150117 10764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:41:02.155805 10764 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:41:02.158730 10764 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8: Loaded TSK: 0
I20260810 14:41:02.160924 10764 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:41:02.163911 10748 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260810 14:41:02.570319 10750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:02.570811 10750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:02.570992 10750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:02.583003 10750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:02.583221 10750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:02.583452 10750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:41:02.600270 10750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.55.84:49927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:38335
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=41323
--tserver_master_addrs=127.21.55.126:33111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:02.602164 10750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:02.603993 10750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:02.659175 10807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:02.663151 10805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:02.665438 10810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:02.666940 10750 server_base.cc:1034] running on GCE node
I20260810 14:41:02.668226 10750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:02.670248 10750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:02.671588 10750 hybrid_clock.cc:648] HybridClock initialized: now 1786372862671435 us; error 126 us; skew 500 ppm
I20260810 14:41:02.672040 10750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:02 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372862, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:02.677166 10750 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:41:02.698073 10750 webserver.cc:460] Webserver started at http://127.21.55.65:41323/ using document root <none> and password file <none>
I20260810 14:41:02.699219 10750 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:02.699429 10750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:41:02 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372862, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:41:02.727725 10750 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 14:41:02.745663 10824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:02.747900 10750 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.008s sys 0.008s
I20260810 14:41:02.748198 10750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "dbe4f06c736c4c15861bfbb51bcad2d3"
format_stamp: "Formatted at 2026-08-10 14:40:58 on dist-test-slave-h6n0"
server_key: "a51f6c99c165bdae503ff96bb3a123ba"
server_key_iv: "c1e2afd485812e6a1a8f68a468f12bfe"
server_key_version: "kuduclusterkey@0"
I20260810 14:41:02.749732 10750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:02.807412 10750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:02.812192 10750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:02.812562 10750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:02.813756 10750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:02.815644 10750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:02.815762 10750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:02.815869 10750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:02.815968 10750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:02.878903 10750 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:38335
I20260810 14:41:02.878957 11012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:38335 every 8 connection(s)
I20260810 14:41:02.880411 10750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:41:02.889458 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 10750
I20260810 14:41:02.890316 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:49927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:34405
--local_ip_for_outbound_sockets=127.21.55.66
--tserver_master_addrs=127.21.55.126:33111
--webserver_port=40751
--webserver_interface=127.21.55.66
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.55.84:49927/kms with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:41:02 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372862, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:02.932380 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:02.884100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:54909 (local address 127.21.55.126:33111)
0810 14:41:02.900365 (+ 16265us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:02.900386 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:02.900446 (+ 60us) server_negotiation.cc:407] Connection header received
0810 14:41:02.902711 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:02.902729 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:02.903193 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:02.905909 (+ 2716us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:02.906345 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:02.908395 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:02.910930 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:02.915873 (+ 4943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:02.916117 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:02.921638 (+ 5521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:02.921651 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:02.921671 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:02.921720 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:02.926159 (+ 4439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:02.927304 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:02.927314 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:02.927321 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:02.927540 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:02.928201 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:02.928209 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:02.928214 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:02.928960 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:02.929151 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:02.930297 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0810 14:41:02.930807 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15985,"thread_start_us":16230,"threads_started":1}
I20260810 14:41:02.935432 11017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:02.884211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33111 (local address 127.21.55.65:54909)
0810 14:41:02.899377 (+ 15166us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:02.899434 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:02.901883 (+ 2449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:02.903816 (+ 1933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:02.903828 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:02.904714 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:02.905928 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:02.905955 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:02.908638 (+ 2683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:02.908644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:02.910706 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:02.910723 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:02.916179 (+ 5456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:02.916188 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:02.916504 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:02.918452 (+ 1948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:02.918486 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:02.921267 (+ 2781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:02.926373 (+ 5106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:02.926385 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:02.926402 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:02.927107 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:02.927716 (+ 609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:02.927723 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:02.927727 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:02.928020 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:02.929263 (+ 1243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:02.929271 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:02.929698 (+ 427us) client_negotiation.cc:769] Sending connection context
0810 14:41:02.933412 (+ 3714us) client_negotiation.cc:240] Negotiation successful
0810 14:41:02.934139 (+ 727us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14830,"thread_start_us":14629,"threads_started":1}
I20260810 14:41:02.940006 11013 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33111
I20260810 14:41:02.940344 11013 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:02.941310 11013 heartbeater.cc:511] Master 127.21.55.126:33111 requested a full tablet report, sending...
W20260810 14:41:02.945111 11013 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:33111 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 14:41:02.945353 11013 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:02.945659 11013 heartbeater.cc:511] Master 127.21.55.126:33111 requested a full tablet report, sending...
I20260810 14:41:02.947089 11013 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:02.947432 11013 heartbeater.cc:511] Master 127.21.55.126:33111 requested a full tablet report, sending...
W20260810 14:41:02.948701 11013 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:41:03.321285 11020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:03.321712 11020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:03.321933 11020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:03.334594 11020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:03.334764 11020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:03.335042 11020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:41:03.354118 11020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.55.84:49927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:34405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=40751
--tserver_master_addrs=127.21.55.126:33111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:03.356247 11020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:03.358142 11020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:03.391304 11048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:03.391912 11049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:03.393088 11054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:03.393963 11020 server_base.cc:1034] running on GCE node
I20260810 14:41:03.394771 11020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:03.396088 11020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:03.397352 11020 hybrid_clock.cc:648] HybridClock initialized: now 1786372863397270 us; error 65 us; skew 500 ppm
I20260810 14:41:03.397766 11020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:03 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372863, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:03.402871 11020 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:41:03.427656 11020 webserver.cc:460] Webserver started at http://127.21.55.66:40751/ using document root <none> and password file <none>
I20260810 14:41:03.428841 11020 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:03.429041 11020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:41:03 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372863, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:41:03.459065 11020 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:41:03.463724 11063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:03.465575 11020 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260810 14:41:03.465862 11020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "954ae02bcec54f67a8ce7ac04e344959"
format_stamp: "Formatted at 2026-08-10 14:40:59 on dist-test-slave-h6n0"
server_key: "cb943078ee68812eb8f633675933cecc"
server_key_iv: "78ec41eaf6a98f7bab3e8452552fc18e"
server_key_version: "kuduclusterkey@0"
I20260810 14:41:03.467739 11020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 10 14:41:03 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786372863, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:03.498324 11020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:03.505362 11020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:03.505828 11020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:03.510272 11020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:03.512524 11020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:03.512738 11020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 14:41:03.512914 11020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:03.513042 11020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:03.564355 11020 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:34405
I20260810 14:41:03.564464 11251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:34405 every 8 connection(s)
I20260810 14:41:03.566322 11020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:41:03.569069 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 11020
I20260810 14:41:03.570104 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.55.84:49927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:42849
--local_ip_for_outbound_sockets=127.21.55.67
--tserver_master_addrs=127.21.55.126:33111
--webserver_port=43105
--webserver_interface=127.21.55.67
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.55.84:49927/kms with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:41:03 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372863, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:03.610230 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:03.581151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:48883 (local address 127.21.55.126:33111)
0810 14:41:03.583717 (+ 2566us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:03.583724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:03.586633 (+ 2909us) server_negotiation.cc:407] Connection header received
0810 14:41:03.587695 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:03.587702 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:03.587790 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:03.587893 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:03.590172 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:03.591326 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:03.593653 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:03.594287 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:03.594517 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:03.599938 (+ 5421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:03.599946 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:03.599951 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:03.599996 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:03.603788 (+ 3792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:03.605708 (+ 1920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:03.605718 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:03.605722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:03.605994 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:03.606913 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:03.606920 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:03.606924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:03.607453 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:03.607622 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:03.609272 (+ 1650us) server_negotiation.cc:299] Negotiation successful
0810 14:41:03.609619 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2420,"thread_start_us":2282,"threads_started":1}
I20260810 14:41:03.613152 11257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:03.580505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33111 (local address 127.21.55.66:48883)
0810 14:41:03.584455 (+ 3950us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:03.584510 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:03.587221 (+ 2711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:03.588393 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:03.588409 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:03.589134 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:03.589946 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:03.589972 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:03.591537 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:03.591542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:03.593483 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:03.593499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:03.594936 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:03.594943 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:03.595234 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:03.596091 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:03.596128 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:03.599586 (+ 3458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:03.604151 (+ 4565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:03.604162 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:03.604183 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:03.605392 (+ 1209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:03.606234 (+ 842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:03.606255 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:03.606260 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:03.606727 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:03.607927 (+ 1200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:03.607940 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:03.608623 (+ 683us) client_negotiation.cc:769] Sending connection context
0810 14:41:03.610843 (+ 2220us) client_negotiation.cc:240] Negotiation successful
0810 14:41:03.611604 (+ 761us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3631,"thread_start_us":3262,"threads_started":1}
I20260810 14:41:03.615752 11253 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33111
I20260810 14:41:03.616098 11253 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:03.617004 11253 heartbeater.cc:511] Master 127.21.55.126:33111 requested a full tablet report, sending...
I20260810 14:41:03.619191 10692 ts_manager.cc:194] Registered new tserver with Master: 954ae02bcec54f67a8ce7ac04e344959 (127.21.55.66:34405)
I20260810 14:41:03.621385 10692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:48883
I20260810 14:41:03.948907 11013 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:03.949389 11013 heartbeater.cc:511] Master 127.21.55.126:33111 requested a full tablet report, sending...
I20260810 14:41:03.950779 10692 ts_manager.cc:194] Registered new tserver with Master: dbe4f06c736c4c15861bfbb51bcad2d3 (127.21.55.65:38335)
I20260810 14:41:03.951887 10692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:54909
Aug 10 14:41:04 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372863, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
W20260810 14:41:04.102936 11259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:04.103434 11259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:04.103711 11259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:04.116959 11259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:04.117126 11259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:04.117393 11259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:41:04.129464 11259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.55.84:49927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:42849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=43105
--tserver_master_addrs=127.21.55.126:33111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:04.130991 11259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:04.132324 11259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:04.151168 11267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:04.151759 11265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:04.152098 11271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:04.153061 11259 server_base.cc:1034] running on GCE node
I20260810 14:41:04.153810 11259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:04.155057 11259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:04.156559 11259 hybrid_clock.cc:648] HybridClock initialized: now 1786372864156492 us; error 93 us; skew 500 ppm
I20260810 14:41:04.157034 11259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:04 dist-test-slave-h6n0 krb5kdc[9029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372864, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:04.160691 11259 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:41:04.166913 11259 webserver.cc:460] Webserver started at http://127.21.55.67:43105/ using document root <none> and password file <none>
I20260810 14:41:04.167697 11259 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:04.167871 11259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:41:04 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372864, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:41:04.192221 11259 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.003s
I20260810 14:41:04.199491 11281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:04.201165 11259 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.008s sys 0.000s
I20260810 14:41:04.201372 11259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "aa543c17462f41e2bda4fdbf98dea5b4"
format_stamp: "Formatted at 2026-08-10 14:41:00 on dist-test-slave-h6n0"
server_key: "a5e327a064d64b24aeee0b8ca0e6c629"
server_key_iv: "7770f408d1550bba8d2c373236a567c4"
server_key_version: "kuduclusterkey@0"
I20260810 14:41:04.202477 11259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:04.223724 11259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:04.229924 11259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:04.230412 11259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:04.231843 11259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:04.233814 11259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:04.233983 11259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:04.234123 11259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:04.234233 11259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:04.306797 11259 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:42849
I20260810 14:41:04.306859 11465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:42849 every 8 connection(s)
I20260810 14:41:04.308471 11259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:41:04.310657 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 11259
Aug 10 14:41:04 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372864, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:04.377291 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.311826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:40887 (local address 127.21.55.126:33111)
0810 14:41:04.347972 (+ 36146us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:04.347978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:04.348006 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:41:04.350616 (+ 2610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:04.350623 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:04.350806 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:04.350940 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:04.356231 (+ 5291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.357526 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.360091 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.360761 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.360886 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.368595 (+ 7709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:04.368602 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:04.368607 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:04.368651 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:04.372025 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:04.374345 (+ 2320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:04.374351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:04.374354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:04.374509 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:04.375132 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:04.375138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:04.375142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:04.375484 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:04.375619 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:04.376633 (+ 1014us) server_negotiation.cc:299] Negotiation successful
0810 14:41:04.376875 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36005,"thread_start_us":35868,"threads_started":1}
I20260810 14:41:04.381073 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.312748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:33111 (local address 127.21.55.67:40887)
0810 14:41:04.343672 (+ 30924us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:04.343738 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:04.346459 (+ 2721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:04.354296 (+ 7837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:04.354306 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:04.355041 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:04.355976 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.356012 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.357750 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.357754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.359903 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.359920 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.361315 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.361323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.361633 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.363449 (+ 1816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:04.363485 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:04.368323 (+ 4838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:04.372301 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:04.372310 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:04.372329 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:04.374192 (+ 1863us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:04.374686 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:04.374692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:04.374696 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:04.374979 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:04.375703 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:04.375712 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:04.376173 (+ 461us) client_negotiation.cc:769] Sending connection context
0810 14:41:04.378787 (+ 2614us) client_negotiation.cc:240] Negotiation successful
0810 14:41:04.379589 (+ 802us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30414,"thread_start_us":33457,"threads_started":1}
I20260810 14:41:04.383497 11466 heartbeater.cc:347] Connected to a master server at 127.21.55.126:33111
I20260810 14:41:04.383872 11466 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:04.384888 11466 heartbeater.cc:511] Master 127.21.55.126:33111 requested a full tablet report, sending...
I20260810 14:41:04.386823 10692 ts_manager.cc:194] Registered new tserver with Master: aa543c17462f41e2bda4fdbf98dea5b4 (127.21.55.67:42849)
I20260810 14:41:04.387063 11475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.347920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.21.55.126:33111)
0810 14:41:04.367001 (+ 19081us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:04.367005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:04.367023 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:41:04.367066 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:04.367069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:04.367135 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:04.367232 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:04.369508 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.370913 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.372883 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.373664 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.373803 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.377738 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:04.377744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:04.377749 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:04.377789 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:04.381803 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:04.383140 (+ 1337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:04.383148 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:04.383153 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:04.383317 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:04.384056 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:04.384061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:04.384064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:04.384443 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:04.384723 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:04.385882 (+ 1159us) server_negotiation.cc:299] Negotiation successful
0810 14:41:04.386242 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18967,"thread_start_us":18804,"threads_started":1}
I20260810 14:41:04.388142 10692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:40887
Aug 10 14:41:04 dist-test-slave-h6n0 krb5kdc[9029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372863, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for HTTP/127.21.55.84@KRBTEST.COM
I20260810 14:41:04.394454 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:41:04.427487 11475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.405828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54200 (local address 127.21.55.126:33111)
0810 14:41:04.406037 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:04.406045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:04.408646 (+ 2601us) server_negotiation.cc:407] Connection header received
0810 14:41:04.408834 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:04.408840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:04.408930 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:04.409073 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:04.411393 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.412644 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.414821 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.415532 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.415645 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.419919 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:04.419926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:04.419932 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:04.419981 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:04.423820 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:04.424872 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:04.424879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:04.424883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:04.425056 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:04.425554 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:04.425559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:04.425563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:04.425957 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:04.426213 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:04.426855 (+ 642us) server_negotiation.cc:299] Negotiation successful
0810 14:41:04.427120 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:41:04.434103 10692 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:41:04.445163 10692 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:41:04.550948 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.484229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:38335 (local address 127.0.0.1:52878)
0810 14:41:04.523487 (+ 39258us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:04.523518 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:04.523784 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:04.525388 (+ 1604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:04.525399 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:04.525445 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:04.526035 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.526053 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.539060 (+ 13007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.539067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.547091 (+ 8024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.547110 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.549753 (+ 2643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.549760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.549998 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.550055 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 14:41:04.550214 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 14:41:04.550442 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39048,"mutex_wait_us":564,"thread_start_us":29829,"threads_started":1}
I20260810 14:41:04.565213 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.484910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:34405 (local address 127.0.0.1:44322)
0810 14:41:04.529158 (+ 44248us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:04.529191 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:04.529393 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:04.535758 (+ 6365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:04.535765 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:04.535804 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:04.536305 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.536322 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.541938 (+ 5616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.541945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.544345 (+ 2400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.544365 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.564268 (+ 19903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.564275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.564483 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.564522 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 14:41:04.564622 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 14:41:04.564794 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44029,"thread_start_us":43887,"threads_started":1}
I20260810 14:41:04.566289 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.484625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52878 (local address 127.21.55.65:38335)
0810 14:41:04.516428 (+ 31803us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:04.516444 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:04.524002 (+ 7558us) server_negotiation.cc:407] Connection header received
0810 14:41:04.524212 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:04.524223 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:04.524633 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:04.524950 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:04.526304 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.538842 (+ 12538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.547358 (+ 8516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.549155 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.549314 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.549582 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:04.565665 (+ 16083us) server_negotiation.cc:299] Negotiation successful
0810 14:41:04.565909 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31508,"thread_start_us":47733,"threads_started":1}
I20260810 14:41:04.571295 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.485328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44322 (local address 127.21.55.66:34405)
0810 14:41:04.516939 (+ 31611us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:04.516947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:04.529612 (+ 12665us) server_negotiation.cc:407] Connection header received
0810 14:41:04.529758 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:04.529768 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:04.530120 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:04.530351 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:04.536604 (+ 6253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.538434 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.544594 (+ 6160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.546268 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.546429 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.546711 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:04.570682 (+ 23971us) server_negotiation.cc:299] Negotiation successful
0810 14:41:04.570895 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31455,"thread_start_us":31301,"threads_started":1}
I20260810 14:41:04.570828 10899 tablet_service.cc:1467] Processing CreateTablet for tablet 1286951e48174b8ea3d114ef3996a3d6 (DEFAULT_TABLE table=test-table [id=dd69379c0eec4058a7cd4831e3cc2eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:04.572397 10899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1286951e48174b8ea3d114ef3996a3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:04.577610 11147 tablet_service.cc:1467] Processing CreateTablet for tablet 1286951e48174b8ea3d114ef3996a3d6 (DEFAULT_TABLE table=test-table [id=dd69379c0eec4058a7cd4831e3cc2eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:04.579435 11147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1286951e48174b8ea3d114ef3996a3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:04.579468 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.484229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:42849 (local address 127.0.0.1:37844)
0810 14:41:04.513750 (+ 29521us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:04.513844 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:04.514258 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:04.561123 (+ 46865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:04.561130 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:04.561169 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:04.561565 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.561581 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.572661 (+ 11080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.572669 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.575165 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.575191 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.578482 (+ 3291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.578493 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.578745 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.578787 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 14:41:04.578873 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 14:41:04.579086 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28980,"thread_start_us":38756,"threads_started":1}
I20260810 14:41:04.580669 11501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.484911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37844 (local address 127.21.55.67:42849)
0810 14:41:04.551895 (+ 66984us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:04.551905 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:04.551946 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:41:04.552059 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:04.552069 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:04.552363 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:04.561821 (+ 9458us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:04.562165 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.563982 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.576486 (+ 12504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.578207 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.579750 (+ 1543us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.580005 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:04.580098 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 14:41:04.580279 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66758,"thread_start_us":66559,"threads_started":1}
I20260810 14:41:04.584636 11366 tablet_service.cc:1467] Processing CreateTablet for tablet 1286951e48174b8ea3d114ef3996a3d6 (DEFAULT_TABLE table=test-table [id=dd69379c0eec4058a7cd4831e3cc2eda]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:04.587666 11366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1286951e48174b8ea3d114ef3996a3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:04.613479 11505 tablet_bootstrap.cc:492] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3: Bootstrap starting.
I20260810 14:41:04.620627 11506 tablet_bootstrap.cc:492] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: Bootstrap starting.
I20260810 14:41:04.622737 11505 tablet_bootstrap.cc:654] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:04.625478 11508 tablet_bootstrap.cc:492] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4: Bootstrap starting.
I20260810 14:41:04.626449 11253 heartbeater.cc:503] Master 127.21.55.126:33111 was elected leader, sending a full tablet report...
I20260810 14:41:04.628150 11505 log.cc:824] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:04.629395 11506 tablet_bootstrap.cc:654] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:04.630843 11508 tablet_bootstrap.cc:654] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:04.630841 11506 log.cc:824] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:04.632292 11508 log.cc:824] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:04.637363 11506 tablet_bootstrap.cc:492] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: No bootstrap required, opened a new log
I20260810 14:41:04.641127 11505 tablet_bootstrap.cc:492] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3: No bootstrap required, opened a new log
I20260810 14:41:04.641839 11505 ts_tablet_manager.cc:1397] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.009s
I20260810 14:41:04.646948 11506 ts_tablet_manager.cc:1397] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.005s
I20260810 14:41:04.655580 11508 tablet_bootstrap.cc:492] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4: No bootstrap required, opened a new log
I20260810 14:41:04.656112 11508 ts_tablet_manager.cc:1397] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.002s
I20260810 14:41:04.659864 11506 raft_consensus.cc:348] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42849 } } peers { permanent_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 38335 } } peers { permanent_uuid: "954ae02bcec54f67a8ce7ac04e344959" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34405 } }
I20260810 14:41:04.667227 11505 raft_consensus.cc:348] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42849 } } peers { permanent_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 38335 } } peers { permanent_uuid: "954ae02bcec54f67a8ce7ac04e344959" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34405 } }
I20260810 14:41:04.667815 11505 raft_consensus.cc:374] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:04.667994 11505 raft_consensus.cc:729] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe4f06c736c4c15861bfbb51bcad2d3, State: Initialized, Role: FOLLOWER
I20260810 14:41:04.668897 11505 consensus_queue.cc:260] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42849 } } peers { permanent_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 38335 } } peers { permanent_uuid: "954ae02bcec54f67a8ce7ac04e344959" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34405 } }
I20260810 14:41:04.669658 11506 raft_consensus.cc:374] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:04.669854 11506 raft_consensus.cc:729] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 954ae02bcec54f67a8ce7ac04e344959, State: Initialized, Role: FOLLOWER
I20260810 14:41:04.670624 11506 consensus_queue.cc:260] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42849 } } peers { permanent_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 38335 } } peers { permanent_uuid: "954ae02bcec54f67a8ce7ac04e344959" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34405 } }
I20260810 14:41:04.679383 11508 raft_consensus.cc:348] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42849 } } peers { permanent_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 38335 } } peers { permanent_uuid: "954ae02bcec54f67a8ce7ac04e344959" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34405 } }
I20260810 14:41:04.680038 11508 raft_consensus.cc:374] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:04.680192 11508 raft_consensus.cc:729] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa543c17462f41e2bda4fdbf98dea5b4, State: Initialized, Role: FOLLOWER
I20260810 14:41:04.681025 11508 consensus_queue.cc:260] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42849 } } peers { permanent_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 38335 } } peers { permanent_uuid: "954ae02bcec54f67a8ce7ac04e344959" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34405 } }
I20260810 14:41:04.682508 11505 ts_tablet_manager.cc:1428] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3: Time spent starting tablet: real 0.040s user 0.014s sys 0.006s
I20260810 14:41:04.682961 11013 heartbeater.cc:503] Master 127.21.55.126:33111 was elected leader, sending a full tablet report...
I20260810 14:41:04.689714 11506 ts_tablet_manager.cc:1428] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: Time spent starting tablet: real 0.043s user 0.014s sys 0.004s
I20260810 14:41:04.691355 11466 heartbeater.cc:503] Master 127.21.55.126:33111 was elected leader, sending a full tablet report...
I20260810 14:41:04.691960 11508 ts_tablet_manager.cc:1428] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4: Time spent starting tablet: real 0.036s user 0.009s sys 0.009s
I20260810 14:41:04.712471 11517 raft_consensus.cc:482] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:41:04.712852 11517 raft_consensus.cc:504] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42849 } } peers { permanent_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 38335 } } peers { permanent_uuid: "954ae02bcec54f67a8ce7ac04e344959" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34405 } }
I20260810 14:41:04.721266 11517 leader_election.cc:288] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbe4f06c736c4c15861bfbb51bcad2d3 (127.21.55.65:38335), 954ae02bcec54f67a8ce7ac04e344959 (127.21.55.66:34405)
I20260810 14:41:04.723824 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.715461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:38335 (local address 127.21.55.67:43383)
0810 14:41:04.715955 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:04.715976 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:04.716104 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:04.717553 (+ 1449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:04.717558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:04.717588 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:04.717806 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.717819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.719281 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.719285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.721316 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.721336 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.723082 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.723088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.723278 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.723312 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 14:41:04.723404 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 14:41:04.723551 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352}
I20260810 14:41:04.724650 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.716414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:43383 (local address 127.21.55.65:38335)
0810 14:41:04.716839 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:04.716857 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:04.716879 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:41:04.716959 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:04.716964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:04.717040 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:04.717136 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:04.717995 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.719098 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.721541 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.722754 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.722918 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.722993 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:04.724270 (+ 1277us) server_negotiation.cc:299] Negotiation successful
0810 14:41:04.724415 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278}
I20260810 14:41:04.726118 10936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1286951e48174b8ea3d114ef3996a3d6" candidate_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" is_pre_election: true
I20260810 14:41:04.727327 10936 raft_consensus.cc:2454] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa543c17462f41e2bda4fdbf98dea5b4 in term 0.
I20260810 14:41:04.728369 11282 leader_election.cc:302] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa543c17462f41e2bda4fdbf98dea5b4, dbe4f06c736c4c15861bfbb51bcad2d3; no voters:
I20260810 14:41:04.730934 11517 raft_consensus.cc:2781] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:41:04.731211 11517 raft_consensus.cc:482] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:41:04.731405 11517 raft_consensus.cc:3037] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:04.734717 11517 raft_consensus.cc:504] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42849 } } peers { permanent_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 38335 } } peers { permanent_uuid: "954ae02bcec54f67a8ce7ac04e344959" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34405 } }
I20260810 14:41:04.736043 10936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1286951e48174b8ea3d114ef3996a3d6" candidate_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3"
I20260810 14:41:04.736279 10936 raft_consensus.cc:3037] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:04.740442 10936 raft_consensus.cc:2454] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa543c17462f41e2bda4fdbf98dea5b4 in term 1.
I20260810 14:41:04.741345 11282 leader_election.cc:302] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa543c17462f41e2bda4fdbf98dea5b4, dbe4f06c736c4c15861bfbb51bcad2d3; no voters:
I20260810 14:41:04.742230 11517 leader_election.cc:288] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [CANDIDATE]: Term 1 election: Requested vote from peers dbe4f06c736c4c15861bfbb51bcad2d3 (127.21.55.65:38335), 954ae02bcec54f67a8ce7ac04e344959 (127.21.55.66:34405)
I20260810 14:41:04.742553 11517 raft_consensus.cc:2781] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:41:04.742648 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.716620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:36065 (local address 127.21.55.66:34405)
0810 14:41:04.717463 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:04.717469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:04.726597 (+ 9128us) server_negotiation.cc:407] Connection header received
0810 14:41:04.726707 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:04.726713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:04.726796 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:04.726893 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:04.735574 (+ 8681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.736790 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.739660 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.740986 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.741183 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.741286 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:04.742131 (+ 845us) server_negotiation.cc:299] Negotiation successful
0810 14:41:04.742300 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694}
I20260810 14:41:04.742777 11517 raft_consensus.cc:686] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 1 LEADER]: Becoming Leader. State: Replica: aa543c17462f41e2bda4fdbf98dea5b4, State: Running, Role: LEADER
I20260810 14:41:04.743671 11518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.715574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:34405 (local address 127.21.55.67:36065)
0810 14:41:04.726395 (+ 10821us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:04.726417 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:04.726583 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:04.734777 (+ 8194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:04.734784 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:04.734823 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:04.735282 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.735298 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.737058 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.737064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.739418 (+ 2354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:04.739437 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.741471 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.741480 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:04.741862 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.741911 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 14:41:04.743035 (+ 1124us) client_negotiation.cc:240] Negotiation successful
0810 14:41:04.743184 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10697,"thread_start_us":11077,"threads_started":1}
I20260810 14:41:04.743410 11517 consensus_queue.cc:237] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42849 } } peers { permanent_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 38335 } } peers { permanent_uuid: "954ae02bcec54f67a8ce7ac04e344959" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34405 } }
I20260810 14:41:04.744932 11183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1286951e48174b8ea3d114ef3996a3d6" candidate_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954ae02bcec54f67a8ce7ac04e344959" is_pre_election: true
I20260810 14:41:04.745536 11183 raft_consensus.cc:2454] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa543c17462f41e2bda4fdbf98dea5b4 in term 0.
I20260810 14:41:04.746090 11179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1286951e48174b8ea3d114ef3996a3d6" candidate_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954ae02bcec54f67a8ce7ac04e344959"
I20260810 14:41:04.746326 11179 raft_consensus.cc:3037] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:04.751329 11179 raft_consensus.cc:2454] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa543c17462f41e2bda4fdbf98dea5b4 in term 1.
I20260810 14:41:04.755820 10692 catalog_manager.cc:5495] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa543c17462f41e2bda4fdbf98dea5b4 (127.21.55.67). New cstate: current_term: 1 leader_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa543c17462f41e2bda4fdbf98dea5b4" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 42849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 38335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "954ae02bcec54f67a8ce7ac04e344959" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34405 } health_report { overall_health: UNKNOWN } } }
I20260810 14:41:04.792526 11501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.784728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37852 (local address 127.21.55.67:42849)
0810 14:41:04.785216 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:04.785224 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:04.785253 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:41:04.785381 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:04.785386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:04.785519 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:04.785717 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:41:04.786316 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.787712 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.789687 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.790277 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.790414 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.791101 (+ 687us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:41:04.791875 (+ 774us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:41:04.792045 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:04.792143 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 14:41:04.792235 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 14:41:04.802060 10936 raft_consensus.cc:1261] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3 [term 1 FOLLOWER]: Refusing update from remote peer aa543c17462f41e2bda4fdbf98dea5b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:41:04.802152 11179 raft_consensus.cc:1261] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959 [term 1 FOLLOWER]: Refusing update from remote peer aa543c17462f41e2bda4fdbf98dea5b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:41:04.803145 11523 consensus_queue.cc:1035] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "954ae02bcec54f67a8ce7ac04e344959" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:41:04.803859 11517 consensus_queue.cc:1035] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe4f06c736c4c15861bfbb51bcad2d3" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 38335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:41:04.817586 11533 mvcc.cc:205] Tried to move back new op lower bound from 7316983254218711040 to 7316983254002528256. Current Snapshot: MvccSnapshot[applied={T|T < 7316983254218711040}]
W20260810 14:41:04.824878 11255 tablet.cc:2369] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:41:04.837406 11534 mvcc.cc:205] Tried to move back new op lower bound from 7316983254218711040 to 7316983254002528256. Current Snapshot: MvccSnapshot[applied={T|T < 7316983254218711040}]
I20260810 14:41:04.847307 10692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:41:04.847854 10692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:41:04.854640 10692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e69e54aeea0415d918982183dbfa2d8: Sending DeleteTablet for 3 replicas of tablet 1286951e48174b8ea3d114ef3996a3d6
I20260810 14:41:04.856253 11366 tablet_service.cc:1514] Processing DeleteTablet for tablet 1286951e48174b8ea3d114ef3996a3d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:04 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:37844
I20260810 14:41:04.856899 11147 tablet_service.cc:1514] Processing DeleteTablet for tablet 1286951e48174b8ea3d114ef3996a3d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:04 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:44322
I20260810 14:41:04.859061 10899 tablet_service.cc:1514] Processing DeleteTablet for tablet 1286951e48174b8ea3d114ef3996a3d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:04 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:52878
I20260810 14:41:04.866859 11145 tablet_service.cc:1514] Processing DeleteTablet for tablet 1286951e48174b8ea3d114ef3996a3d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:04 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:44322
I20260810 14:41:04.867129 11547 tablet_replica.cc:330] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4: stopping tablet replica
I20260810 14:41:04.874300 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 10750
I20260810 14:41:04.876197 11549 tablet_replica.cc:330] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3: stopping tablet replica
I20260810 14:41:04.876765 11549 raft_consensus.cc:2229] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:41:04.877225 11549 raft_consensus.cc:2258] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:04.884548 11547 raft_consensus.cc:2229] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:41:04.893782 11548 tablet_replica.cc:330] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: stopping tablet replica
I20260810 14:41:04.894727 11547 raft_consensus.cc:2258] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:04.895735 11548 raft_consensus.cc:2229] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:41:04.896661 11548 raft_consensus.cc:2258] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:04.896770 11549 ts_tablet_manager.cc:1905] T 1286951e48174b8ea3d114ef3996a3d6 P dbe4f06c736c4c15861bfbb51bcad2d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:04.898518 11547 ts_tablet_manager.cc:1905] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:04.899205 11548 ts_tablet_manager.cc:1905] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:04.905666 11547 ts_tablet_manager.cc:1918] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:41:04.906088 11547 log.cc:1196] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/1286951e48174b8ea3d114ef3996a3d6
I20260810 14:41:04.907586 11547 ts_tablet_manager.cc:1939] T 1286951e48174b8ea3d114ef3996a3d6 P aa543c17462f41e2bda4fdbf98dea5b4: Deleting consensus metadata
I20260810 14:41:04.908625 11548 ts_tablet_manager.cc:1918] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:41:04.910261 10669 catalog_manager.cc:4841] TS aa543c17462f41e2bda4fdbf98dea5b4 (127.21.55.67:42849): tablet 1286951e48174b8ea3d114ef3996a3d6 (table test-table [id=dd69379c0eec4058a7cd4831e3cc2eda]) successfully deleted
I20260810 14:41:04.910749 11548 log.cc:1196] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/1286951e48174b8ea3d114ef3996a3d6
I20260810 14:41:04.911355 11548 ts_tablet_manager.cc:1939] T 1286951e48174b8ea3d114ef3996a3d6 P 954ae02bcec54f67a8ce7ac04e344959: Deleting consensus metadata
I20260810 14:41:04.912221 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:04.879065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52892 (local address 127.21.55.65:38335)
0810 14:41:04.879699 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:04.879706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:04.879724 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:41:04.879785 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:04.879791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:04.880059 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:04.880184 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:04.883359 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.884556 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.886589 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:04.887804 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:04.887935 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:04.892272 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:04.892288 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:04.892309 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:04.892440 (+ 131us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:04.907788 (+ 15348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:04.909522 (+ 1734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:04.909538 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:04.909542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:04.909762 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:04.910281 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:04.910288 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:04.910299 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:04.910898 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:04.911243 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:04.911522 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 14:41:04.911768 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505}
I20260810 14:41:04.912964 11007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:04.916390 10671 catalog_manager.cc:4841] TS 954ae02bcec54f67a8ce7ac04e344959 (127.21.55.66:34405): tablet 1286951e48174b8ea3d114ef3996a3d6 (table test-table [id=dd69379c0eec4058a7cd4831e3cc2eda]) successfully deleted
W20260810 14:41:04.917621 10671 catalog_manager.cc:4808] TS 954ae02bcec54f67a8ce7ac04e344959 (127.21.55.66:34405): delete failed for tablet 1286951e48174b8ea3d114ef3996a3d6 because the tablet was not found. No further retry: Not found: Tablet not found: 1286951e48174b8ea3d114ef3996a3d6
I20260810 14:41:05.085078 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 10750
W20260810 14:41:05.117372 10668 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.55.65:38335 (error 108)
I20260810 14:41:05.117619 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 11020
I20260810 14:41:05.141753 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:05.119190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.21.55.66:34405)
0810 14:41:05.119366 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:05.119372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:05.119403 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:41:05.119471 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:05.119477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:05.119555 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:05.119662 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:05.122488 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:05.123964 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:05.126669 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:05.127383 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:05.127526 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:05.131604 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:05.131627 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:05.131651 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:05.131706 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:05.136770 (+ 5064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:05.137823 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:05.137833 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:05.137837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:05.138018 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:05.138480 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:05.138485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:05.138489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:05.139376 (+ 887us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:05.139722 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:05.141039 (+ 1317us) server_negotiation.cc:299] Negotiation successful
0810 14:41:05.141342 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:41:05.142740 11240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:05.157828 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:05.119329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:38335 (local address 127.0.0.1:52902)
0810 14:41:05.156362 (+ 37033us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:05.157569 (+ 1207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36847,"thread_start_us":36705,"threads_started":1}
W20260810 14:41:05.158320 10668 catalog_manager.cc:4584] TS dbe4f06c736c4c15861bfbb51bcad2d3 (127.21.55.65:38335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1286951e48174b8ea3d114ef3996a3d6: Network error: Client connection negotiation failed: client connection to 127.21.55.65:38335: connect: Connection refused (error 111)
I20260810 14:41:05.212404 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:05.211811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:38335 (local address 127.0.0.1:52918)
0810 14:41:05.212120 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:05.212229 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260810 14:41:05.213506 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:05.213152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:38335 (local address 127.0.0.1:52922)
0810 14:41:05.213291 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:05.213377 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 14:41:05.274003 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:05.273504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:38335 (local address 127.0.0.1:52930)
0810 14:41:05.273732 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:05.273845 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 14:41:05.275264 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:05.274885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:38335 (local address 127.0.0.1:52944)
0810 14:41:05.275052 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:05.275134 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 14:41:05.312167 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 11020
I20260810 14:41:05.347054 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 11259
I20260810 14:41:05.356110 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:05.355512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:38335 (local address 127.0.0.1:52948)
0810 14:41:05.355817 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:05.355919 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260810 14:41:05.357266 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:05.356870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:38335 (local address 127.0.0.1:52954)
0810 14:41:05.357082 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:05.357149 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 14:41:05.367084 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:05.347992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37864 (local address 127.21.55.67:42849)
0810 14:41:05.349283 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:05.349290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:05.349313 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:41:05.349374 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:05.349378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:05.349464 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:05.349592 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:05.351553 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:05.352850 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:05.355004 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:05.355454 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:05.355613 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:05.359427 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:05.359438 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:05.359456 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:05.359496 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:05.363701 (+ 4205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:05.364649 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:05.364656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:05.364659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:05.364818 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:05.365297 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:05.365302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:05.365304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:05.365690 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:05.365974 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:05.366382 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 14:41:05.366692 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1177,"thread_start_us":1006,"threads_started":1}
I20260810 14:41:05.367858 11460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:05.496277 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 11259
I20260810 14:41:05.529721 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 10640
I20260810 14:41:05.530385 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:05.529944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:38335 (local address 127.0.0.1:52968)
0810 14:41:05.530148 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:05.530240 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 14:41:05.531865 10737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:05.650669 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:05.531449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:38335 (local address 127.0.0.1:52976)
0810 14:41:05.531750 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:05.531866 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:38335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260810 14:41:05.651391 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 10640
I20260810 14:41:05.689668 21725 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 14:41:06.039813 21725 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 14:41:06.040103 21725 mini_ranger.cc:66] Stopping Ranger...
I20260810 14:41:06.398052 21725 mini_ranger.cc:68] Stopped Ranger
2026-08-10 14:41:06.398 UTC [9084] LOG: received smart shutdown request
2026-08-10 14:41:06.403 UTC [9084] LOG: background worker "logical replication launcher" (PID 9100) exited with exit code 1
2026-08-10 14:41:06.403 UTC [9091] LOG: shutting down
2026-08-10 14:41:06.648 UTC [9084] LOG: database system is shut down
2026-08-10T14:41:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (59401 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:41:06 dist-test-slave-h6n0 krb5kdc[11575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:41:06 dist-test-slave-h6n0 krb5kdc[11575](info): set up 2 sockets
Aug 10 14:41:06 dist-test-slave-h6n0 krb5kdc[11575](info): commencing operation
krb5kdc: starting...
W20260810 14:41:08.321353 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.456s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:41:08 dist-test-slave-h6n0 krb5kdc[11575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:41:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:41:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:41:08.654680 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:34773
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:34773,127.21.55.125:39061,127.21.55.124:39119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:09.034008 11595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:09.034391 11595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:09.034618 11595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:09.042486 11595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:41:09.042722 11595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:09.042846 11595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:09.042958 11595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:41:09.043030 11595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:41:09.053541 11595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:34773,127.21.55.125:39061,127.21.55.124:39119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:34773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:09.055030 11595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:09.056360 11595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:09.070652 11603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:09.070698 11606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:09.070752 11604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:09.071830 11595 server_base.cc:1034] running on GCE node
I20260810 14:41:09.073194 11595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:09.075205 11595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:09.076501 11595 hybrid_clock.cc:648] HybridClock initialized: now 1786372869076432 us; error 96 us; skew 500 ppm
I20260810 14:41:09.076942 11595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:09 dist-test-slave-h6n0 krb5kdc[11575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372869, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:09.081329 11595 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:41:09.083449 11595 webserver.cc:460] Webserver started at http://127.21.55.126:34269/ using document root <none> and password file <none>
I20260810 14:41:09.084228 11595 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:09.084389 11595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:09.084710 11595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:09.087890 11595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "823a098a1a2748508ed4ac8fd35acf32"
format_stamp: "Formatted at 2026-08-10 14:41:09 on dist-test-slave-h6n0"
server_key: "f55e5e6ee18e253dfb306b9397115098"
server_key_iv: "fb9fc2cea98b4cb71de60e4466eec7c3"
server_key_version: "encryptionkey@0"
I20260810 14:41:09.088707 11595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "823a098a1a2748508ed4ac8fd35acf32"
format_stamp: "Formatted at 2026-08-10 14:41:09 on dist-test-slave-h6n0"
server_key: "f55e5e6ee18e253dfb306b9397115098"
server_key_iv: "fb9fc2cea98b4cb71de60e4466eec7c3"
server_key_version: "encryptionkey@0"
I20260810 14:41:09.095108 11595 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 14:41:09.099633 11615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:09.101286 11595 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:41:09.101449 11595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "823a098a1a2748508ed4ac8fd35acf32"
format_stamp: "Formatted at 2026-08-10 14:41:09 on dist-test-slave-h6n0"
server_key: "f55e5e6ee18e253dfb306b9397115098"
server_key_iv: "fb9fc2cea98b4cb71de60e4466eec7c3"
server_key_version: "encryptionkey@0"
I20260810 14:41:09.101640 11595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:09.118038 11595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:09.122496 11595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:09.122876 11595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:09.146971 11595 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:34773
I20260810 14:41:09.147029 11697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:34773 every 8 connection(s)
I20260810 14:41:09.148473 11595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:41:09.152263 11699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:09.157135 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 11595
I20260810 14:41:09.157413 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:41:09.158138 21725 external_mini_cluster.cc:1428] Setting key df747444cba40f17d11a41b9bd3b7ab2
I20260810 14:41:09.159245 11699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:41:09 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372869, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.55.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.125@KRBTEST.COM" created.
I20260810 14:41:09.193991 11703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.162850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36270 (local address 127.21.55.126:34773)
0810 14:41:09.172362 (+ 9512us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:09.172372 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:09.172433 (+ 61us) server_negotiation.cc:407] Connection header received
0810 14:41:09.174879 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:09.174885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:09.175130 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:09.175360 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:09.176673 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.177880 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.179365 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.179916 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.180132 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:09.184754 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:09.184764 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:09.184781 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:09.184817 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:09.188688 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:09.189923 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:09.189931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:09.189934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:09.190158 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:09.190690 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:09.190702 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:09.190704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:09.191303 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:09.191540 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:09.192120 (+ 580us) server_negotiation.cc:299] Negotiation successful
0810 14:41:09.192469 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9215,"spinlock_wait_cycles":315136,"thread_start_us":8936,"threads_started":1}
I20260810 14:41:09.194026 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.162712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:34773 (local address 127.0.0.1:36270)
0810 14:41:09.171244 (+ 8532us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.171312 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:09.174419 (+ 3107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:09.175372 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:09.175379 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:09.176127 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:09.176465 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:09.176482 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.178045 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.178050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:09.179215 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:09.179229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.180186 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.180192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:09.180359 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:09.181288 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:09.181326 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:09.184417 (+ 3091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:09.188881 (+ 4464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:09.188897 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:09.188914 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:09.189628 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:09.190285 (+ 657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:09.190292 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:09.190298 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:09.190569 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:09.191471 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:09.191480 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:09.191723 (+ 243us) client_negotiation.cc:769] Sending connection context
0810 14:41:09.191977 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 14:41:09.192623 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8193,"mutex_wait_us":170,"thread_start_us":7967,"threads_started":1}
I20260810 14:41:09.197314 11699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:41:09.198607 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.198134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52726)
0810 14:41:09.198309 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.198436 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260810 14:41:09.198897 11616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
I20260810 14:41:09.200690 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.200192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52732)
0810 14:41:09.200380 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.200451 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260810 14:41:09.200984 11699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:39061: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab.
Entry for principal kudu/127.21.55.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab.
I20260810 14:41:09.226457 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.125:39061
--webserver_interface=127.21.55.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab
--principal=kudu/127.21.55.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:34773,127.21.55.125:39061,127.21.55.124:39119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:41:09.250501 11699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } attempt: 1
I20260810 14:41:09.251832 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.251376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52742)
0810 14:41:09.251580 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.251701 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 14:41:09.253434 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.252906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52746)
0810 14:41:09.253090 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.253200 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 14:41:09.253710 11699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:39061: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
I20260810 14:41:09.322003 11699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } attempt: 2
I20260810 14:41:09.323307 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.322839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52754)
0810 14:41:09.323055 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.323156 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 14:41:09.324321 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.323950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52762)
0810 14:41:09.324103 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.324190 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:41:09.324517 11699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:39061: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
I20260810 14:41:09.415777 11699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } attempt: 3
I20260810 14:41:09.417032 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.416549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52770)
0810 14:41:09.416777 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.416883 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 14:41:09.418057 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.417687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52776)
0810 14:41:09.417852 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.417918 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 14:41:09.418258 11699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:39061: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
I20260810 14:41:09.561537 11699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } attempt: 4
I20260810 14:41:09.562847 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.562211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52786)
0810 14:41:09.562605 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.562693 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260810 14:41:09.563913 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.563506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52800)
0810 14:41:09.563653 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.563748 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 14:41:09.564095 11699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:39061: Network error: Client connection negotiation failed: client connection to 127.21.55.125:39061: connect: Connection refused (error 111)
W20260810 14:41:09.586700 11707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:09.587203 11707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:09.587386 11707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:09.594898 11707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:41:09.595049 11707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:09.595160 11707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:09.595266 11707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:41:09.595353 11707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:41:09.606299 11707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:34773,127.21.55.125:39061,127.21.55.124:39119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab
--principal=kudu/127.21.55.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.125:39061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.55.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:09.607803 11707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:09.609901 11707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:09.624464 11716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:09.624518 11718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:09.624547 11721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:09.625613 11707 server_base.cc:1034] running on GCE node
I20260810 14:41:09.626497 11707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:09.628353 11707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:09.629733 11707 hybrid_clock.cc:648] HybridClock initialized: now 1786372869629681 us; error 47 us; skew 500 ppm
I20260810 14:41:09.630151 11707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:09 dist-test-slave-h6n0 krb5kdc[11575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372869, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:09.633643 11707 init.cc:377] Logged in from keytab as kudu/127.21.55.125@KRBTEST.COM (short username kudu)
I20260810 14:41:09.635471 11707 webserver.cc:460] Webserver started at http://127.21.55.125:35755/ using document root <none> and password file <none>
I20260810 14:41:09.636166 11707 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:09.636293 11707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:09.636622 11707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:09.639775 11707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/data/instance:
uuid: "cfafea5eca944672aab1dd075153037b"
format_stamp: "Formatted at 2026-08-10 14:41:09 on dist-test-slave-h6n0"
server_key: "1197bcdc8e268d166daf80e48128c72a"
server_key_iv: "c80ff263f4a9ca82706fc0fcca6d9ab4"
server_key_version: "encryptionkey@0"
I20260810 14:41:09.640705 11707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/wal/instance:
uuid: "cfafea5eca944672aab1dd075153037b"
format_stamp: "Formatted at 2026-08-10 14:41:09 on dist-test-slave-h6n0"
server_key: "1197bcdc8e268d166daf80e48128c72a"
server_key_iv: "c80ff263f4a9ca82706fc0fcca6d9ab4"
server_key_version: "encryptionkey@0"
I20260810 14:41:09.646406 11707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:41:09.650995 11738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:09.652916 11707 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 14:41:09.653137 11707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/wal
uuid: "cfafea5eca944672aab1dd075153037b"
format_stamp: "Formatted at 2026-08-10 14:41:09 on dist-test-slave-h6n0"
server_key: "1197bcdc8e268d166daf80e48128c72a"
server_key_iv: "c80ff263f4a9ca82706fc0fcca6d9ab4"
server_key_version: "encryptionkey@0"
I20260810 14:41:09.653363 11707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:09.669423 11707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:09.675325 11707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:09.675721 11707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:09.699586 11707 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.125:39061
I20260810 14:41:09.699638 11821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.125:39061 every 8 connection(s)
I20260810 14:41:09.701058 11707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
I20260810 14:41:09.705266 11822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:09.711104 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 11707
I20260810 14:41:09.711326 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-1/wal/instance
I20260810 14:41:09.712035 21725 external_mini_cluster.cc:1428] Setting key 3bbd96f6a40ca73c4785aaceab02ed00
I20260810 14:41:09.715792 11822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:41:09 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372869, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.55.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.124@KRBTEST.COM" created.
I20260810 14:41:09.748683 11824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.721534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.21.55.126:34773)
0810 14:41:09.722089 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:09.722094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:09.723454 (+ 1360us) server_negotiation.cc:407] Connection header received
0810 14:41:09.726066 (+ 2612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:09.726076 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:09.726191 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:09.726334 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:09.730085 (+ 3751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.731676 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.733609 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.734345 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.734508 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:09.739955 (+ 5447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:09.739961 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:09.739965 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:09.740202 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:09.744439 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:09.745630 (+ 1191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:09.745639 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:09.745645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:09.745866 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:09.746347 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:09.746354 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:09.746360 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:09.746855 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:09.747046 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:09.747854 (+ 808us) server_negotiation.cc:299] Negotiation successful
0810 14:41:09.748162 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":310,"threads_started":1}
I20260810 14:41:09.749322 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.722214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:34773 (local address 127.0.0.1:36272)
0810 14:41:09.723215 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.723280 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:09.725700 (+ 2420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:09.726942 (+ 1242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:09.726957 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:09.728174 (+ 1217us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:09.729674 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:09.729722 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.731972 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.731980 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:09.733410 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:09.733426 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.734763 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.734769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:09.735341 (+ 572us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:09.736283 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:09.736309 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:09.739587 (+ 3278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:09.744665 (+ 5078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:09.744679 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:09.744702 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:09.745448 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:09.746011 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:09.746015 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:09.746017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:09.746212 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:09.747034 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:09.747041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:09.747409 (+ 368us) client_negotiation.cc:769] Sending connection context
0810 14:41:09.747673 (+ 264us) client_negotiation.cc:240] Negotiation successful
0810 14:41:09.748116 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":404,"threads_started":1}
I20260810 14:41:09.751674 11822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:41:09 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372869, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:41:09.771876 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.752577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52808)
0810 14:41:09.752779 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.752801 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:09.752922 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:09.754267 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:09.754270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:09.754752 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:09.754899 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:09.754908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.756720 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.756727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:09.758406 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:09.758425 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.759866 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.759880 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:09.760130 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:09.761087 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:09.761117 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:09.764058 (+ 2941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:09.768908 (+ 4850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:09.768915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:09.768919 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:09.769510 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:09.769950 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:09.769957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:09.769962 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:09.770120 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:09.770852 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:09.770861 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:09.771006 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 14:41:09.771151 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 14:41:09.771464 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 14:41:09.771999 11830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.752749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.21.55.125:39061)
0810 14:41:09.753730 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:09.753740 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:09.753776 (+ 36us) server_negotiation.cc:407] Connection header received
0810 14:41:09.753843 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:09.753854 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:09.754088 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:09.754259 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:09.755086 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.756542 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.758810 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.759618 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.759782 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:09.764318 (+ 4536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:09.764333 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:09.764356 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:09.764387 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:09.768527 (+ 4140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:09.769672 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:09.769684 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:09.769693 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:09.769824 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:09.770237 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:09.770244 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:09.770248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:09.770715 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:09.770943 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:09.771327 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 14:41:09.771593 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":746,"thread_start_us":577,"threads_started":1}
I20260810 14:41:09.774389 11822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:41:09.775696 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.775198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47474)
0810 14:41:09.775401 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.775535 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 14:41:09.776006 11744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
I20260810 14:41:09.777981 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.777229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47478)
0810 14:41:09.777555 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.777804 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 14:41:09.778345 11822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:39119: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab.
Entry for principal kudu/127.21.55.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab.
I20260810 14:41:09.783025 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.124:39119
--webserver_interface=127.21.55.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab
--principal=kudu/127.21.55.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:34773,127.21.55.125:39061,127.21.55.124:39119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:41:09.827853 11822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } attempt: 1
I20260810 14:41:09.829309 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.828704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47482)
0810 14:41:09.828959 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.829101 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 14:41:09.831065 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.830495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47492)
0810 14:41:09.830807 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.830907 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
W20260810 14:41:09.831516 11822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:39119: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
I20260810 14:41:09.863360 11699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } attempt: 5
Aug 10 14:41:09 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372869, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:41:09.880910 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.864258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52822)
0810 14:41:09.864638 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.864662 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:09.864839 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:09.865307 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:09.865311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:09.865773 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:09.865979 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:09.865989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.867420 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.867424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:09.868402 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:09.868414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.869411 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.869416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:09.869592 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:09.870302 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:09.870320 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:09.872874 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:09.878144 (+ 5270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:09.878148 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:09.878151 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:09.878640 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:09.879214 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:09.879218 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:09.879221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:09.879380 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:09.880083 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:09.880087 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:09.880235 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 14:41:09.880394 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 14:41:09.880629 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255}
I20260810 14:41:09.881081 11830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.864425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.21.55.125:39061)
0810 14:41:09.864628 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:09.864635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:09.864775 (+ 140us) server_negotiation.cc:407] Connection header received
0810 14:41:09.865012 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:09.865020 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:09.865116 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:09.865252 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:09.866168 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.867270 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.868552 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:09.869257 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:09.869388 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:09.873091 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:09.873097 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:09.873102 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:09.873141 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:09.877941 (+ 4800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:09.878766 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:09.878772 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:09.878775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:09.878941 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:09.879520 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:09.879527 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:09.879531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:09.879927 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:09.880083 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:09.880542 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 14:41:09.880778 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:41:09.882289 11699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:41:09.883459 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.883042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47506)
0810 14:41:09.883240 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.883317 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 14:41:09.884407 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.884065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47512)
0810 14:41:09.884224 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.884310 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 14:41:09.884682 11699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:39119: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
I20260810 14:41:09.899966 11822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } attempt: 2
I20260810 14:41:09.901605 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.900863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47528)
0810 14:41:09.901182 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.901347 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260810 14:41:09.903097 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.902684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47544)
0810 14:41:09.902839 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.902944 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 14:41:09.903355 11822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:39119: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
I20260810 14:41:09.935972 11699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } attempt: 1
I20260810 14:41:09.937337 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.936765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47556)
0810 14:41:09.936996 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.937132 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 14:41:09.939039 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.938098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47572)
0810 14:41:09.938480 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.938783 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 14:41:09.939440 11699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:39119: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
I20260810 14:41:09.994642 11822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } attempt: 3
I20260810 14:41:09.996079 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.995453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47574)
0810 14:41:09.995714 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.995847 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 14:41:09.997272 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:09.996827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47590)
0810 14:41:09.997047 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:09.997141 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260810 14:41:09.997562 11822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:39119: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
I20260810 14:41:10.007721 11699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } attempt: 2
I20260810 14:41:10.009099 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.008422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47604)
0810 14:41:10.008705 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:10.008832 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 14:41:10.010512 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.010078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47606)
0810 14:41:10.010282 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:10.010370 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260810 14:41:10.011147 11699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:39119: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
I20260810 14:41:10.117484 11699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } attempt: 3
I20260810 14:41:10.118811 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.118190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47614)
0810 14:41:10.118437 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:10.118587 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 14:41:10.120380 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.119967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47616)
0810 14:41:10.120204 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:10.120279 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260810 14:41:10.120680 11699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:39119: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
I20260810 14:41:10.140822 11822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } attempt: 4
I20260810 14:41:10.142380 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.141632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47624)
0810 14:41:10.142066 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:10.142172 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 14:41:10.143501 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.143118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47626)
0810 14:41:10.143275 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:10.143371 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:41:10.143733 11822 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:39119: Network error: Client connection negotiation failed: client connection to 127.21.55.124:39119: connect: Connection refused (error 111)
W20260810 14:41:10.156766 11835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:10.157114 11835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:10.157210 11835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:10.165556 11835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:41:10.165678 11835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:10.165725 11835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:10.165786 11835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:41:10.165830 11835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:41:10.178558 11835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:34773,127.21.55.125:39061,127.21.55.124:39119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab
--principal=kudu/127.21.55.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.124:39119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.55.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:10.180054 11835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:10.181370 11835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:10.194667 11841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:10.194679 11849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:10.195343 11843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:10.195731 11835 server_base.cc:1034] running on GCE node
I20260810 14:41:10.196599 11835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:10.198010 11835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:10.199214 11835 hybrid_clock.cc:648] HybridClock initialized: now 1786372870199174 us; error 37 us; skew 500 ppm
I20260810 14:41:10.199587 11835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:10 dist-test-slave-h6n0 krb5kdc[11575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372870, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:10.202994 11835 init.cc:377] Logged in from keytab as kudu/127.21.55.124@KRBTEST.COM (short username kudu)
I20260810 14:41:10.204532 11835 webserver.cc:460] Webserver started at http://127.21.55.124:41255/ using document root <none> and password file <none>
I20260810 14:41:10.205166 11835 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:10.205268 11835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:10.205577 11835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:10.208602 11835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/data/instance:
uuid: "1b4e1f134af548d683217d5d2a0c269a"
format_stamp: "Formatted at 2026-08-10 14:41:10 on dist-test-slave-h6n0"
server_key: "806d58bb75d393cf33fe7feb9febe23e"
server_key_iv: "09ebf628babd1568ac3393a4aaedf1ec"
server_key_version: "encryptionkey@0"
I20260810 14:41:10.209359 11835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/wal/instance:
uuid: "1b4e1f134af548d683217d5d2a0c269a"
format_stamp: "Formatted at 2026-08-10 14:41:10 on dist-test-slave-h6n0"
server_key: "806d58bb75d393cf33fe7feb9febe23e"
server_key_iv: "09ebf628babd1568ac3393a4aaedf1ec"
server_key_version: "encryptionkey@0"
I20260810 14:41:10.214929 11835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:41:10.218439 11859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:10.220211 11835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:41:10.220404 11835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/wal
uuid: "1b4e1f134af548d683217d5d2a0c269a"
format_stamp: "Formatted at 2026-08-10 14:41:10 on dist-test-slave-h6n0"
server_key: "806d58bb75d393cf33fe7feb9febe23e"
server_key_iv: "09ebf628babd1568ac3393a4aaedf1ec"
server_key_version: "encryptionkey@0"
I20260810 14:41:10.220897 11835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:10.255098 11835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:10.260231 11835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:10.260576 11835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:10.288800 11835 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.124:39119
I20260810 14:41:10.288864 11937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.124:39119 every 8 connection(s)
I20260810 14:41:10.291365 11835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
I20260810 14:41:10.294200 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 11835
I20260810 14:41:10.294806 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786372641103011-21725-0/minicluster-data/master-2/wal/instance
I20260810 14:41:10.295488 21725 external_mini_cluster.cc:1428] Setting key aa4772915ff9b9e519d455c1b5c1c814
I20260810 14:41:10.297775 11939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:10.297957 11699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } attempt: 4
I20260810 14:41:10.309453 11939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:41:10 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372869, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
Aug 10 14:41:10 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
I20260810 14:41:10.323413 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.298674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47652)
0810 14:41:10.298849 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:10.298873 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:10.298999 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:10.304327 (+ 5328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:10.304336 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:10.305311 (+ 975us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:10.305574 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:10.305592 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.307221 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.307226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:10.308129 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:10.308141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.309621 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.309628 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:10.309865 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.310863 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:10.310890 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:10.313358 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:10.319028 (+ 5670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:10.319035 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:10.319040 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:10.319905 (+ 865us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:10.320611 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:10.320617 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:10.320624 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:10.320827 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:10.322111 (+ 1284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:10.322121 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:10.322366 (+ 245us) client_negotiation.cc:769] Sending connection context
0810 14:41:10.322516 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 14:41:10.322907 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 14:41:10.326663 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.299017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47652 (local address 127.21.55.124:39119)
0810 14:41:10.299859 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:10.299865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:10.299889 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:41:10.300920 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:10.300952 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:10.301652 (+ 700us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:10.302112 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:10.305872 (+ 3760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.307069 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.308283 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.309420 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.309656 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.313762 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:10.313778 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:10.313801 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:10.313863 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:10.318804 (+ 4941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.320144 (+ 1340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.320154 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.320165 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.320444 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.321021 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.321025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.321029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.321925 (+ 896us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:10.323750 (+ 1825us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:10.324284 (+ 534us) server_negotiation.cc:299] Negotiation successful
0810 14:41:10.324857 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":691,"spinlock_wait_cycles":26368,"thread_start_us":494,"threads_started":1}
I20260810 14:41:10.326717 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.297659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.21.55.124:39119)
0810 14:41:10.298706 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:10.298722 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:10.298771 (+ 49us) server_negotiation.cc:407] Connection header received
0810 14:41:10.300920 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:10.300952 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:10.301652 (+ 700us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:10.302112 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:10.304275 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.306460 (+ 2185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.308636 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.309421 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.309663 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.316100 (+ 6437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:10.316109 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:10.316120 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:10.316182 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:10.320449 (+ 4267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.321504 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.321509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.321512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.321672 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.322210 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.322215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.322219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.322638 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:10.324663 (+ 2025us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:10.324857 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 14:41:10.325053 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":177,"server-negotiator.queue_time_us":748,"spinlock_wait_cycles":57216,"thread_start_us":487,"threads_started":1}
I20260810 14:41:10.335419 11699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32: Bootstrap starting.
Aug 10 14:41:10 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372870, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:10.340732 11699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:10.342442 11699 log.cc:824] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:41:10 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:10.347983 11699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32: No bootstrap required, opened a new log
I20260810 14:41:10.352296 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.313097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:34773 (local address 127.0.0.1:36288)
0810 14:41:10.324649 (+ 11552us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:10.324710 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:10.324874 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:10.326287 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:10.326295 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:10.327328 (+ 1033us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:10.327717 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:10.327732 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.329108 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.329114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:10.335389 (+ 6275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:10.335405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.336557 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.336564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:10.336790 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.338789 (+ 1999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:10.338821 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:10.341792 (+ 2971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:10.348529 (+ 6737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:10.348552 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:10.348562 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:10.349357 (+ 795us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:10.349875 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:10.349881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:10.349886 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:10.350112 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:10.351157 (+ 1045us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:10.351166 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:10.351395 (+ 229us) client_negotiation.cc:769] Sending connection context
0810 14:41:10.351670 (+ 275us) client_negotiation.cc:240] Negotiation successful
0810 14:41:10.351984 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11291,"thread_start_us":11088,"threads_started":1}
I20260810 14:41:10.352617 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.313056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36288 (local address 127.21.55.126:34773)
0810 14:41:10.325961 (+ 12905us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:10.325966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:10.325984 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:41:10.326030 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:10.326034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:10.326097 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:10.326234 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:10.327901 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.328904 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.335548 (+ 6644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.336308 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.336461 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.342179 (+ 5718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:10.342190 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:10.342194 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:10.342241 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:10.348262 (+ 6021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.349540 (+ 1278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.349545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.349549 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.349709 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.350409 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.350422 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.350425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.350975 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:10.351173 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:10.351817 (+ 644us) server_negotiation.cc:299] Negotiation successful
0810 14:41:10.352117 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12585,"thread_start_us":12391,"threads_started":1}
I20260810 14:41:10.354038 11939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:41:10.356182 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.331223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.21.55.126:34773)
0810 14:41:10.332194 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:10.332201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:10.332227 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:41:10.332280 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:10.332285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:10.332380 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:10.332511 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:10.334030 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.335471 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.337192 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.337814 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.337960 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.345007 (+ 7047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:10.345018 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:10.345025 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:10.345072 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:10.349340 (+ 4268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.351028 (+ 1688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.351034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.351038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.351216 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.353247 (+ 2031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.353252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.353256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.353672 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:10.353979 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:10.355631 (+ 1652us) server_negotiation.cc:299] Negotiation successful
0810 14:41:10.355885 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827,"thread_start_us":548,"threads_started":1}
I20260810 14:41:10.360822 11699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } }
I20260810 14:41:10.361377 11699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:10.361553 11699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 823a098a1a2748508ed4ac8fd35acf32, State: Initialized, Role: FOLLOWER
I20260810 14:41:10.362344 11699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } }
Aug 10 14:41:10 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372870, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:41:10.364748 11953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } } }
I20260810 14:41:10.365288 11953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:41:10.366412 11699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 14:41:10 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:41:10.374141 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.354906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:39061 (local address 127.0.0.1:52824)
0810 14:41:10.355194 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:10.355210 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:10.355346 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:10.355742 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:10.355749 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:10.356289 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:10.356501 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:10.356514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.358163 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.358169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:10.359771 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:10.359789 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.361128 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.361135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:10.361337 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.362187 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:10.362212 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:10.365631 (+ 3419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:10.370854 (+ 5223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:10.370862 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:10.370869 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:10.371535 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:10.372050 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:10.372056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:10.372061 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:10.372264 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:10.373032 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:10.373038 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:10.373219 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 14:41:10.373395 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 14:41:10.373739 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260810 14:41:10.374673 11830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.354910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.21.55.125:39061)
0810 14:41:10.355098 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:10.355105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:10.355290 (+ 185us) server_negotiation.cc:407] Connection header received
0810 14:41:10.355485 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:10.355490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:10.355570 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:10.355673 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:10.356696 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.357912 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.359949 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.360919 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.361082 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.365869 (+ 4787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:10.365881 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:10.365887 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:10.365930 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:10.370671 (+ 4741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.371712 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.371719 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.371724 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.371899 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.372420 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.372427 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.372431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.372794 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:10.372959 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:10.373672 (+ 713us) server_negotiation.cc:299] Negotiation successful
0810 14:41:10.374033 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:41:10.375772 11939 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } has no permanent_uuid. Determining permanent_uuid...
W20260810 14:41:10.376255 11632 tablet.cc:2369] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:41:10.382882 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.357963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.21.55.125:39061)
0810 14:41:10.363043 (+ 5080us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:10.363047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:10.363067 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:41:10.363156 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:10.363161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:10.363230 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:10.363346 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:10.365005 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.366112 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.367735 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.368505 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.368664 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.374694 (+ 6030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:10.374702 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:10.374707 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:10.374740 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:10.378599 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.379553 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.379559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.379562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.379705 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.380181 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.380187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.380191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.380582 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:10.381970 (+ 1388us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:10.382095 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 14:41:10.382368 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4986,"thread_start_us":4767,"threads_started":1}
Aug 10 14:41:10 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372870, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
W20260810 14:41:10.386929 11970 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:41:10.387137 11970 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:41:10.390079 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:41:10.393661 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.376648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47668)
0810 14:41:10.377409 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:10.377429 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:10.377551 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:10.377973 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:10.377978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:10.378493 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:10.378743 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:10.378756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.380097 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.380104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:10.381458 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:10.381475 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.382572 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.382580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:10.382762 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.383396 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:10.383420 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:10.386406 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:10.390731 (+ 4325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:10.390736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:10.390739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:10.391395 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:10.391806 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:10.391811 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:10.391814 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:10.391954 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:10.392605 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:10.392610 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:10.392768 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 14:41:10.392918 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 14:41:10.393268 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637}
I20260810 14:41:10.396373 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.377209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47668 (local address 127.21.55.124:39119)
0810 14:41:10.377467 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:10.377476 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:10.377501 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:41:10.377684 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:10.377693 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:10.377790 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:10.377924 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:10.378881 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.379937 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.381611 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.382220 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.382323 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.386653 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:10.386661 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:10.386667 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:10.386703 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:10.390600 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.391550 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.391556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.391560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.391698 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.392060 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.392064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.392068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.392471 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:10.394625 (+ 2154us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:10.394806 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 14:41:10.395619 (+ 813us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 14:41:10.384308 11595 master_runner.cc:186] Error getting master registration for 127.21.55.125:39061: OK, instance_id { permanent_uuid: "cfafea5eca944672aab1dd075153037b" instance_seqno: 1786372869692642 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:41:10.400436 11595 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:41:10.405983 11939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a: Bootstrap starting.
I20260810 14:41:10.415436 11939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:10.418802 11939 log.cc:824] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:10.444429 11822 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } attempt: 5
I20260810 14:41:10.448344 11939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a: No bootstrap required, opened a new log
I20260810 14:41:10.451323 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.411849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52854 (local address 127.21.55.125:39061)
0810 14:41:10.412705 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:10.412715 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:10.419502 (+ 6787us) server_negotiation.cc:407] Connection header received
0810 14:41:10.419796 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:10.419812 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:10.419980 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:10.420399 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:10.424404 (+ 4005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.426054 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.430026 (+ 3972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.431031 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.431167 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.437068 (+ 5901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:10.437079 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:10.437088 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:10.437154 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:10.447405 (+ 10251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.448500 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.448506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.448510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.448683 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.449241 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.449247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.449251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.449620 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:10.449888 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:10.450712 (+ 824us) server_negotiation.cc:299] Negotiation successful
0810 14:41:10.450974 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656}
Aug 10 14:41:10 dist-test-slave-h6n0 krb5kdc[11575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372869, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
I20260810 14:41:10.470690 11939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } }
I20260810 14:41:10.471325 11939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:10.471531 11939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4e1f134af548d683217d5d2a0c269a, State: Initialized, Role: FOLLOWER
I20260810 14:41:10.472371 11939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } }
I20260810 14:41:10.473752 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.410754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36314 (local address 127.21.55.126:34773)
0810 14:41:10.411015 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:10.411028 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:10.427467 (+ 16439us) server_negotiation.cc:407] Connection header received
0810 14:41:10.427809 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:10.427815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:10.427908 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:10.428005 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:10.438730 (+ 10725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.440756 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.450728 (+ 9972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.451741 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.452167 (+ 426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.462686 (+ 10519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:10.462697 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:10.462703 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:10.462780 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:10.467533 (+ 4753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.469110 (+ 1577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.469118 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.469122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.469314 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.470365 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.470371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.470375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.471033 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:10.471531 (+ 498us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:10.472955 (+ 1424us) server_negotiation.cc:299] Negotiation successful
0810 14:41:10.473265 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 14:41:10.479034 11939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:41:10.480513 11826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.445542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:39119 (local address 127.0.0.1:47678)
0810 14:41:10.446078 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:10.446104 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:10.446250 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:10.447405 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:10.447411 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:10.447956 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:10.448238 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:10.448255 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.449952 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.449958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:10.451561 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:10.451577 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.454290 (+ 2713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.454297 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:10.454581 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.455239 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:10.455267 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:10.458393 (+ 3126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:10.475640 (+ 17247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:10.475651 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:10.475668 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:10.476351 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:10.477796 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:10.477802 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:10.477805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:10.478293 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:10.479231 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:10.479241 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:10.479470 (+ 229us) client_negotiation.cc:769] Sending connection context
0810 14:41:10.479643 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 14:41:10.480051 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370}
I20260810 14:41:10.481549 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.427681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.21.55.124:39119)
0810 14:41:10.427917 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:10.427923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:10.452606 (+ 24683us) server_negotiation.cc:407] Connection header received
0810 14:41:10.452861 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:10.452866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:10.452937 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:10.453010 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:10.458564 (+ 5554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.459771 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.464010 (+ 4239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.464907 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.465077 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.469590 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:10.469599 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:10.469607 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:10.469666 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:10.474358 (+ 4692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.478351 (+ 3993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.478361 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.478367 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.478639 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.479331 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.479336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.479340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.479697 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:10.480061 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:10.480789 (+ 728us) server_negotiation.cc:299] Negotiation successful
0810 14:41:10.481116 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 14:41:10.482450 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:10.446573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.21.55.124:39119)
0810 14:41:10.446778 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:10.446786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:10.446818 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:41:10.446898 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:10.446905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:10.447006 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:10.447143 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:10.448470 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.449634 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.452086 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:10.452727 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:10.452845 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:10.458653 (+ 5808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:10.458661 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:10.458673 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:10.458798 (+ 125us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:10.474705 (+ 15907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.477380 (+ 2675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.477386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.477390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.477642 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:10.478585 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:10.478590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:10.478594 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:10.479055 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:10.481287 (+ 2232us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:10.481421 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 14:41:10.482074 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:41:10.477334 11984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } } }
I20260810 14:41:10.484947 11984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:41:10.489406 11822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b: Bootstrap starting.
I20260810 14:41:10.493642 11822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:10.495180 11822 log.cc:824] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:10.502151 11822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b: No bootstrap required, opened a new log
W20260810 14:41:10.512933 11869 tablet.cc:2369] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:41:10.515630 11822 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } }
I20260810 14:41:10.516359 11822 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:10.516583 11822 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfafea5eca944672aab1dd075153037b, State: Initialized, Role: FOLLOWER
I20260810 14:41:10.517586 11822 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } }
I20260810 14:41:10.529381 12007 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } } }
I20260810 14:41:10.529914 12007 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:41:10.531049 11822 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [sys.catalog]: configured and running, proceeding with master startup.
W20260810 14:41:10.531276 12004 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:41:10.531421 12004 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:41:10.516978 11835 master_runner.cc:186] Error getting master registration for 127.21.55.125:39061: OK, instance_id { permanent_uuid: "cfafea5eca944672aab1dd075153037b" instance_seqno: 1786372869692642 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:41:10.534698 11835 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:41:10.557142 11707 master_runner.cc:263] No leader master found from master cfafea5eca944672aab1dd075153037b
I20260810 14:41:10.557325 11707 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 14:41:10.569864 12022 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:41:10.570588 12022 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:41:10.616463 11953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:41:10.616832 11953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } }
I20260810 14:41:10.618682 11953 leader_election.cc:288] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfafea5eca944672aab1dd075153037b (127.21.55.125:39061), 1b4e1f134af548d683217d5d2a0c269a (127.21.55.124:39119)
I20260810 14:41:10.619967 11901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "823a098a1a2748508ed4ac8fd35acf32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4e1f134af548d683217d5d2a0c269a" is_pre_election: true
I20260810 14:41:10.620666 11901 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 823a098a1a2748508ed4ac8fd35acf32 in term 0.
I20260810 14:41:10.621348 11625 leader_election.cc:302] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4e1f134af548d683217d5d2a0c269a, 823a098a1a2748508ed4ac8fd35acf32; no voters:
I20260810 14:41:10.623356 11780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "823a098a1a2748508ed4ac8fd35acf32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfafea5eca944672aab1dd075153037b" is_pre_election: true
I20260810 14:41:10.624037 11780 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 823a098a1a2748508ed4ac8fd35acf32 in term 0.
I20260810 14:41:10.626341 11953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:41:10.626514 11953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:41:10.626657 11953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:10.630940 11953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } }
I20260810 14:41:10.632226 11780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "823a098a1a2748508ed4ac8fd35acf32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfafea5eca944672aab1dd075153037b"
I20260810 14:41:10.632475 11780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:10.636272 11780 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 823a098a1a2748508ed4ac8fd35acf32 in term 1.
I20260810 14:41:10.636772 11616 leader_election.cc:302] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 823a098a1a2748508ed4ac8fd35acf32, cfafea5eca944672aab1dd075153037b; no voters:
I20260810 14:41:10.641105 11901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "823a098a1a2748508ed4ac8fd35acf32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4e1f134af548d683217d5d2a0c269a"
I20260810 14:41:10.641381 11901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:10.646159 11901 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 823a098a1a2748508ed4ac8fd35acf32 in term 1.
I20260810 14:41:10.651481 11953 leader_election.cc:288] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [CANDIDATE]: Term 1 election: Requested vote from peers cfafea5eca944672aab1dd075153037b (127.21.55.125:39061), 1b4e1f134af548d683217d5d2a0c269a (127.21.55.124:39119)
I20260810 14:41:10.651770 11953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:41:10.651961 11953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [term 1 LEADER]: Becoming Leader. State: Replica: 823a098a1a2748508ed4ac8fd35acf32, State: Running, Role: LEADER
I20260810 14:41:10.652669 11953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } }
I20260810 14:41:10.670521 12027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 823a098a1a2748508ed4ac8fd35acf32. Latest consensus state: current_term: 1 leader_uuid: "823a098a1a2748508ed4ac8fd35acf32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } } }
I20260810 14:41:10.672821 12027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [sys.catalog]: This master's current role is: LEADER
W20260810 14:41:10.678864 11746 tablet.cc:2369] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:41:10.680024 12031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:41:10.682461 12031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:41:10.703649 11901 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [term 1 FOLLOWER]: Refusing update from remote peer 823a098a1a2748508ed4ac8fd35acf32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:41:10.705157 12027 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:41:10.716367 11780 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [term 1 FOLLOWER]: Refusing update from remote peer 823a098a1a2748508ed4ac8fd35acf32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:41:10.717391 12027 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:41:10.757256 12007 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 823a098a1a2748508ed4ac8fd35acf32. Latest consensus state: current_term: 1 leader_uuid: "823a098a1a2748508ed4ac8fd35acf32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } } }
I20260810 14:41:10.757640 12007 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:41:10.763698 11984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 823a098a1a2748508ed4ac8fd35acf32. Latest consensus state: current_term: 1 leader_uuid: "823a098a1a2748508ed4ac8fd35acf32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } } }
I20260810 14:41:10.764045 11984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:41:10.768118 12033 mvcc.cc:205] Tried to move back new op lower bound from 7316983278388932608 to 7316983278209462272. Current Snapshot: MvccSnapshot[applied={T|T < 7316983278388932608}]
I20260810 14:41:10.774220 12035 mvcc.cc:205] Tried to move back new op lower bound from 7316983278388932608 to 7316983278209462272. Current Snapshot: MvccSnapshot[applied={T|T < 7316983278388932608}]
I20260810 14:41:10.778434 11953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "823a098a1a2748508ed4ac8fd35acf32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } } }
I20260810 14:41:10.778920 11953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:10.785501 11984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "823a098a1a2748508ed4ac8fd35acf32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } } }
I20260810 14:41:10.785776 11984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:41:10.795871 12027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "823a098a1a2748508ed4ac8fd35acf32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } } }
I20260810 14:41:10.796309 12027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:10.798597 12036 mvcc.cc:205] Tried to move back new op lower bound from 7316983278388932608 to 7316983278209462272. Current Snapshot: MvccSnapshot[applied={T|T < 7316983278388932608}]
I20260810 14:41:10.821979 12031 catalog_manager.cc:1347] Generated new cluster ID: 9f3be77e5cbe41cc9265fbbf941d2a90
I20260810 14:41:10.822157 12031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:41:10.841684 12007 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "823a098a1a2748508ed4ac8fd35acf32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "823a098a1a2748508ed4ac8fd35acf32" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34773 } } peers { permanent_uuid: "cfafea5eca944672aab1dd075153037b" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 39061 } } peers { permanent_uuid: "1b4e1f134af548d683217d5d2a0c269a" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 39119 } } }
I20260810 14:41:10.841992 12007 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b [sys.catalog]: This master's current role is: FOLLOWER
W20260810 14:41:10.843210 11625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.55.124:39119, user_credentials={real_user=slave}} blocked reactor thread for 55922.3us
I20260810 14:41:10.918080 12031 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:41:10.924152 12031 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:41:10.945590 12031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 823a098a1a2748508ed4ac8fd35acf32: Generated new TSK 0
I20260810 14:41:10.950966 12031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:41:11.533680 12004 catalog_manager.cc:1259] Loaded cluster ID: 9f3be77e5cbe41cc9265fbbf941d2a90
I20260810 14:41:11.533797 12004 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a: loading cluster ID for follower catalog manager: success
I20260810 14:41:11.536526 12004 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a: acquiring CA information for follower catalog manager: success
I20260810 14:41:11.538410 12004 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1b4e1f134af548d683217d5d2a0c269a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:41:11.572780 12022 catalog_manager.cc:1259] Loaded cluster ID: 9f3be77e5cbe41cc9265fbbf941d2a90
I20260810 14:41:11.572892 12022 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b: loading cluster ID for follower catalog manager: success
I20260810 14:41:11.575333 12022 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b: acquiring CA information for follower catalog manager: success
I20260810 14:41:11.577080 12022 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cfafea5eca944672aab1dd075153037b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:41:11.607110 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 11595
I20260810 14:41:11.609243 11693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:11.725135 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 11595
I20260810 14:41:11.756688 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 11707
I20260810 14:41:11.757925 11814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:11.864614 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 11707
I20260810 14:41:11.892434 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 11835
I20260810 14:41:11.893620 11932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:12.003144 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 11835
2026-08-10T14:41:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5191 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:41:12 dist-test-slave-h6n0 krb5kdc[12066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:41:12 dist-test-slave-h6n0 krb5kdc[12066](info): set up 2 sockets
Aug 10 14:41:12 dist-test-slave-h6n0 krb5kdc[12066](info): commencing operation
krb5kdc: starting...
W20260810 14:41:13.385869 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.330s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:41:13 dist-test-slave-h6n0 krb5kdc[12066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:41:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:41:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:41:13.686231 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:39783
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:43111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:39783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:14.041991 12084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:14.042290 12084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:14.042382 12084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:14.049722 12084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:41:14.049841 12084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:14.049893 12084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:14.049943 12084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:41:14.049988 12084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:41:14.060249 12084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:39783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:39783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:14.061508 12084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:14.062793 12084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:14.075500 12096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:14.075601 12100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:14.075614 12097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:14.076583 12084 server_base.cc:1034] running on GCE node
I20260810 14:41:14.077445 12084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:14.079008 12084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:14.080279 12084 hybrid_clock.cc:648] HybridClock initialized: now 1786372874080240 us; error 56 us; skew 500 ppm
I20260810 14:41:14.080689 12084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:14 dist-test-slave-h6n0 krb5kdc[12066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372874, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:14.085078 12084 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:41:14.087836 12084 webserver.cc:460] Webserver started at http://127.21.55.126:44475/ using document root <none> and password file <none>
I20260810 14:41:14.088691 12084 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:14.088871 12084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:14.089331 12084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:14.093878 12084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "66efcd9188bb41d3a3b2b4e633be6cd4"
format_stamp: "Formatted at 2026-08-10 14:41:14 on dist-test-slave-h6n0"
server_key: "354fa8eaa74460f4fd85f539ae6a3224"
server_key_iv: "ce03115d7401793a122b561b9ad94c7e"
server_key_version: "encryptionkey@0"
I20260810 14:41:14.094928 12084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "66efcd9188bb41d3a3b2b4e633be6cd4"
format_stamp: "Formatted at 2026-08-10 14:41:14 on dist-test-slave-h6n0"
server_key: "354fa8eaa74460f4fd85f539ae6a3224"
server_key_iv: "ce03115d7401793a122b561b9ad94c7e"
server_key_version: "encryptionkey@0"
I20260810 14:41:14.100960 12084 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 14:41:14.105509 12110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:14.107010 12084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 14:41:14.107200 12084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "66efcd9188bb41d3a3b2b4e633be6cd4"
format_stamp: "Formatted at 2026-08-10 14:41:14 on dist-test-slave-h6n0"
server_key: "354fa8eaa74460f4fd85f539ae6a3224"
server_key_iv: "ce03115d7401793a122b561b9ad94c7e"
server_key_version: "encryptionkey@0"
I20260810 14:41:14.107407 12084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:14.129585 12084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:14.135078 12084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:14.135444 12084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:14.159555 12084 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:39783
I20260810 14:41:14.159621 12185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:39783 every 8 connection(s)
I20260810 14:41:14.161113 12084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:41:14.165452 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 12084
I20260810 14:41:14.165709 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:41:14.166476 21725 external_mini_cluster.cc:1428] Setting key 1f6582c08d6e4aded7afdf138440180e
I20260810 14:41:14.166903 12187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:14.185524 12187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4: Bootstrap starting.
Aug 10 14:41:14 dist-test-slave-h6n0 krb5kdc[12066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:14.189370 12187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:14.190876 12187 log.cc:824] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:14.195263 12187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4: No bootstrap required, opened a new log
I20260810 14:41:14.197464 12190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:14.169087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57868 (local address 127.21.55.126:39783)
0810 14:41:14.170176 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:14.170196 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:14.170264 (+ 68us) server_negotiation.cc:407] Connection header received
0810 14:41:14.173241 (+ 2977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:14.173288 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:14.174070 (+ 782us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:14.174752 (+ 682us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:14.177389 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:14.179945 (+ 2556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:14.181591 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:14.182831 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:14.183037 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:14.187637 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:14.187648 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:14.187668 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:14.187774 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:14.192297 (+ 4523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:14.193272 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:14.193284 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:14.193293 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:14.193552 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:14.194018 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:14.194023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:14.194026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:14.194903 (+ 877us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:14.195226 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:14.195886 (+ 660us) server_negotiation.cc:299] Negotiation successful
0810 14:41:14.196429 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":457,"threads_started":1}
I20260810 14:41:14.202832 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:41:14.206354 12187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66efcd9188bb41d3a3b2b4e633be6cd4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39783 } }
I20260810 14:41:14.206934 12187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:14.207129 12187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66efcd9188bb41d3a3b2b4e633be6cd4, State: Initialized, Role: FOLLOWER
I20260810 14:41:14.207764 12187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66efcd9188bb41d3a3b2b4e633be6cd4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39783 } }
I20260810 14:41:14.207981 12187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:41:14.208200 12187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:41:14.208379 12187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:14.210722 12187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66efcd9188bb41d3a3b2b4e633be6cd4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39783 } }
I20260810 14:41:14.211273 12187 leader_election.cc:302] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66efcd9188bb41d3a3b2b4e633be6cd4; no voters:
I20260810 14:41:14.212034 12187 leader_election.cc:288] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:41:14.212178 12194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:41:14.212996 12194 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [term 1 LEADER]: Becoming Leader. State: Replica: 66efcd9188bb41d3a3b2b4e633be6cd4, State: Running, Role: LEADER
I20260810 14:41:14.213563 12194 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66efcd9188bb41d3a3b2b4e633be6cd4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39783 } }
I20260810 14:41:14.214167 12187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:41:14.216552 12196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66efcd9188bb41d3a3b2b4e633be6cd4. Latest consensus state: current_term: 1 leader_uuid: "66efcd9188bb41d3a3b2b4e633be6cd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66efcd9188bb41d3a3b2b4e633be6cd4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39783 } } }
I20260810 14:41:14.216526 12195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66efcd9188bb41d3a3b2b4e633be6cd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66efcd9188bb41d3a3b2b4e633be6cd4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 39783 } } }
I20260810 14:41:14.216969 12195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:14.216969 12196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:14.218869 12203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:41:14.226812 12203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:41:14.236176 12203 catalog_manager.cc:1347] Generated new cluster ID: 68296a6beeeb484fa31f7cefc59cb96b
I20260810 14:41:14.236307 12203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:41:14.254994 12203 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:41:14.256897 12203 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:41:14.266023 12203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66efcd9188bb41d3a3b2b4e633be6cd4: Generated new TSK 0
I20260810 14:41:14.267063 12203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:41:14.318213 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 12084
I20260810 14:41:14.319689 12177 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:14.417372 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 12084
2026-08-10T14:41:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2403 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:41:14 dist-test-slave-h6n0 krb5kdc[12230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:41:14 dist-test-slave-h6n0 krb5kdc[12230](info): set up 2 sockets
Aug 10 14:41:14 dist-test-slave-h6n0 krb5kdc[12230](info): commencing operation
krb5kdc: starting...
W20260810 14:41:15.805861 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.347s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:41:16 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:41:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:41:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:41:16.116735 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38449
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:38449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:16.519544 12249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:16.520054 12249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:16.520412 12249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:16.529337 12249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:41:16.529575 12249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:16.529704 12249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:16.529821 12249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:41:16.529927 12249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:41:16.540479 12249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:38449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:16.541971 12249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:16.543385 12249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:16.556941 12255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:16.557102 12260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:16.556903 12257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:16.559005 12249 server_base.cc:1034] running on GCE node
I20260810 14:41:16.560213 12249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:16.562301 12249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:16.563715 12249 hybrid_clock.cc:648] HybridClock initialized: now 1786372876563638 us; error 70 us; skew 500 ppm
I20260810 14:41:16.564200 12249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:16 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372876, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:16.567826 12249 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:41:16.570132 12249 webserver.cc:460] Webserver started at http://127.21.55.126:45413/ using document root <none> and password file <none>
I20260810 14:41:16.571139 12249 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:16.571264 12249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:16.571548 12249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:16.575162 12249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "d0eabdf2f3bb49d3991048f5981a7882"
format_stamp: "Formatted at 2026-08-10 14:41:16 on dist-test-slave-h6n0"
server_key: "afa95c2cbd4349e855aedc7a7ca75d9c"
server_key_iv: "c67b546f0ba3aeb73c8b81bd4036e037"
server_key_version: "encryptionkey@0"
I20260810 14:41:16.575902 12249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "d0eabdf2f3bb49d3991048f5981a7882"
format_stamp: "Formatted at 2026-08-10 14:41:16 on dist-test-slave-h6n0"
server_key: "afa95c2cbd4349e855aedc7a7ca75d9c"
server_key_iv: "c67b546f0ba3aeb73c8b81bd4036e037"
server_key_version: "encryptionkey@0"
I20260810 14:41:16.581360 12249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:41:16.585421 12268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:16.587129 12249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:41:16.587327 12249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "d0eabdf2f3bb49d3991048f5981a7882"
format_stamp: "Formatted at 2026-08-10 14:41:16 on dist-test-slave-h6n0"
server_key: "afa95c2cbd4349e855aedc7a7ca75d9c"
server_key_iv: "c67b546f0ba3aeb73c8b81bd4036e037"
server_key_version: "encryptionkey@0"
I20260810 14:41:16.587598 12249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:16.627030 12249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:16.632823 12249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:16.633190 12249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:16.657272 12354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:38449 every 8 connection(s)
I20260810 14:41:16.657393 12249 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:38449
I20260810 14:41:16.659246 12249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:41:16.660005 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 12249
I20260810 14:41:16.660315 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:41:16.661362 21725 external_mini_cluster.cc:1428] Setting key 85837606976963c27f84f650568d77b6
I20260810 14:41:16.664003 12355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 14:41:16 dist-test-slave-h6n0 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:16.678213 12355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Bootstrap starting.
I20260810 14:41:16.681800 12355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:16.683260 12355 log.cc:824] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:16.687887 12355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: No bootstrap required, opened a new log
I20260810 14:41:16.689059 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:16.663779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44686 (local address 127.21.55.126:38449)
0810 14:41:16.664652 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:16.664666 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:16.664710 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:41:16.666509 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:16.666589 (+ 80us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:16.667029 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:16.667464 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:16.669138 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:16.670667 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:16.672173 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:16.673181 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:16.673472 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:16.679085 (+ 5613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:16.679093 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:16.679108 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:16.679150 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:16.682441 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:16.683463 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:16.683478 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:16.683484 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:16.683699 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:16.684365 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:16.684373 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:16.684377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:16.685583 (+ 1206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:16.685974 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:16.686929 (+ 955us) server_negotiation.cc:299] Negotiation successful
0810 14:41:16.687593 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":402,"threads_started":1}
I20260810 14:41:16.700284 12355 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:16.700712 12355 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:16.700841 12355 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0eabdf2f3bb49d3991048f5981a7882, State: Initialized, Role: FOLLOWER
I20260810 14:41:16.701479 12355 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:16.701740 12355 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:41:16.701915 12355 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:41:16.702066 12355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:16.704533 12355 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:16.705114 12355 leader_election.cc:302] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0eabdf2f3bb49d3991048f5981a7882; no voters:
I20260810 14:41:16.705762 12355 leader_election.cc:288] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:41:16.705929 12364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:41:16.706684 12364 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 1 LEADER]: Becoming Leader. State: Replica: d0eabdf2f3bb49d3991048f5981a7882, State: Running, Role: LEADER
I20260810 14:41:16.707235 12364 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:16.707996 12355 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:41:16.711499 12367 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0eabdf2f3bb49d3991048f5981a7882. Latest consensus state: current_term: 1 leader_uuid: "d0eabdf2f3bb49d3991048f5981a7882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } } }
I20260810 14:41:16.711582 12366 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0eabdf2f3bb49d3991048f5981a7882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } } }
I20260810 14:41:16.712343 12366 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:16.714660 12367 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:16.715634 12376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:41:16.722114 12376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:41:16.733037 12376 catalog_manager.cc:1347] Generated new cluster ID: 5106100e13c44ac88cfaec46a1f922b9
I20260810 14:41:16.733326 12376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:41:16.746217 12376 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:41:16.747643 12376 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:41:16.772284 12376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Generated new TSK 0
I20260810 14:41:16.773420 12376 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:41:16.868816 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38449
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:17.257382 12395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:17.257758 12395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:17.257925 12395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:17.265640 12395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:17.265748 12395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:17.265987 12395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:41:17.276871 12395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:17.278215 12395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:17.279695 12395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:17.294313 12406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:17.294401 12410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:17.294404 12408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:17.300522 12395 server_base.cc:1034] running on GCE node
I20260810 14:41:17.301445 12395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:17.302879 12395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:17.304157 12395 hybrid_clock.cc:648] HybridClock initialized: now 1786372877304084 us; error 67 us; skew 500 ppm
I20260810 14:41:17.304672 12395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:17 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372877, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:17.314450 12395 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:41:17.316395 12395 webserver.cc:460] Webserver started at http://127.21.55.65:36389/ using document root <none> and password file <none>
I20260810 14:41:17.317245 12395 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:17.317394 12395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:17.317770 12395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:17.320996 12395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "fe3aaa29a27343b6b9a6ab82892f22fe"
format_stamp: "Formatted at 2026-08-10 14:41:17 on dist-test-slave-h6n0"
server_key: "b16eb5c5f25cf51487f635ddea6ffafc"
server_key_iv: "6cd8fe03b09749364dc53e9231169e1d"
server_key_version: "encryptionkey@0"
I20260810 14:41:17.321826 12395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "fe3aaa29a27343b6b9a6ab82892f22fe"
format_stamp: "Formatted at 2026-08-10 14:41:17 on dist-test-slave-h6n0"
server_key: "b16eb5c5f25cf51487f635ddea6ffafc"
server_key_iv: "6cd8fe03b09749364dc53e9231169e1d"
server_key_version: "encryptionkey@0"
I20260810 14:41:17.327932 12395 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 14:41:17.332679 12419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:17.335404 12395 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 14:41:17.335734 12395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "fe3aaa29a27343b6b9a6ab82892f22fe"
format_stamp: "Formatted at 2026-08-10 14:41:17 on dist-test-slave-h6n0"
server_key: "b16eb5c5f25cf51487f635ddea6ffafc"
server_key_iv: "6cd8fe03b09749364dc53e9231169e1d"
server_key_version: "encryptionkey@0"
I20260810 14:41:17.336066 12395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:17.354635 12395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:17.359597 12395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:17.359999 12395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:17.361145 12395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:17.362695 12395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:17.362814 12395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:17.362998 12395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:17.363087 12395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:17.408802 12395 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:45615
I20260810 14:41:17.408856 12593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:45615 every 8 connection(s)
I20260810 14:41:17.410288 12395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:41:17.412691 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 12395
I20260810 14:41:17.412954 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:41:17.413784 21725 external_mini_cluster.cc:1428] Setting key 9b449fefd876df3eaddc1ff7c045d0d6
Aug 10 14:41:17 dist-test-slave-h6n0 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372877, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:17.435644 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:17.413740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:47463 (local address 127.21.55.126:38449)
0810 14:41:17.415059 (+ 1319us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:17.415070 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:17.415671 (+ 601us) server_negotiation.cc:407] Connection header received
0810 14:41:17.417773 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:17.417779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:17.417893 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:17.418021 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:17.419721 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:17.420522 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:17.422272 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:17.422784 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:17.422938 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:17.426983 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:17.426989 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:17.426993 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:17.427029 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:17.431718 (+ 4689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:17.432617 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:17.432625 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:17.432632 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:17.432792 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:17.433265 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:17.433273 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:17.433279 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:17.433717 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:17.433899 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:17.434820 (+ 921us) server_negotiation.cc:299] Negotiation successful
0810 14:41:17.435112 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1106,"thread_start_us":958,"threads_started":1}
I20260810 14:41:17.436498 12598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:17.414305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:47463)
0810 14:41:17.415508 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:17.415549 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:17.417504 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:17.418267 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:17.418276 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:17.418925 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:17.419538 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:17.419557 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:17.420689 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:17.420702 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:17.422090 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:17.422104 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:17.423049 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:17.423060 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:17.423315 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:17.424089 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:17.424113 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:17.426715 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:17.431901 (+ 5186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:17.431908 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:17.431922 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:17.432455 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:17.432915 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:17.432918 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:17.432921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:17.433117 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:17.433901 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:17.433908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:17.434294 (+ 386us) client_negotiation.cc:769] Sending connection context
0810 14:41:17.434673 (+ 379us) client_negotiation.cc:240] Negotiation successful
0810 14:41:17.435147 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"thread_start_us":721,"threads_started":1}
I20260810 14:41:17.438354 12595 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38449
I20260810 14:41:17.438750 12595 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:17.439752 12595 heartbeater.cc:511] Master 127.21.55.126:38449 requested a full tablet report, sending...
I20260810 14:41:17.443050 12298 ts_manager.cc:194] Registered new tserver with Master: fe3aaa29a27343b6b9a6ab82892f22fe (127.21.55.65:45615)
I20260810 14:41:17.444598 12298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:47463
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:41:17.491441 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38449
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:17.878130 12601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:17.878512 12601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:17.878759 12601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:17.888523 12601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:17.888736 12601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:17.888944 12601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:41:17.900131 12601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:17.901508 12601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:17.903030 12601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:17.917740 12611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:17.917865 12614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:17.918329 12610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:17.919108 12601 server_base.cc:1034] running on GCE node
I20260810 14:41:17.919771 12601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:17.920854 12601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:17.922134 12601 hybrid_clock.cc:648] HybridClock initialized: now 1786372877922072 us; error 51 us; skew 500 ppm
I20260810 14:41:17.922678 12601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:17 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372877, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:17.926046 12601 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:41:17.927798 12601 webserver.cc:460] Webserver started at http://127.21.55.66:33569/ using document root <none> and password file <none>
I20260810 14:41:17.928480 12601 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:17.928607 12601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:17.928867 12601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:17.931829 12601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "c833c6dff13c470a8a4e16a88062202f"
format_stamp: "Formatted at 2026-08-10 14:41:17 on dist-test-slave-h6n0"
server_key: "671000839c2305d07ec6e3dd40927c9e"
server_key_iv: "d1fcdfd0759c7f95519f18a60efe2cf2"
server_key_version: "encryptionkey@0"
I20260810 14:41:17.932559 12601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "c833c6dff13c470a8a4e16a88062202f"
format_stamp: "Formatted at 2026-08-10 14:41:17 on dist-test-slave-h6n0"
server_key: "671000839c2305d07ec6e3dd40927c9e"
server_key_iv: "d1fcdfd0759c7f95519f18a60efe2cf2"
server_key_version: "encryptionkey@0"
I20260810 14:41:17.938726 12601 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260810 14:41:17.942971 12622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:17.944912 12601 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 14:41:17.945147 12601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "c833c6dff13c470a8a4e16a88062202f"
format_stamp: "Formatted at 2026-08-10 14:41:17 on dist-test-slave-h6n0"
server_key: "671000839c2305d07ec6e3dd40927c9e"
server_key_iv: "d1fcdfd0759c7f95519f18a60efe2cf2"
server_key_version: "encryptionkey@0"
I20260810 14:41:17.945385 12601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:17.957304 12601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:17.962152 12601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:17.962472 12601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:17.963670 12601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:17.965529 12601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:17.965857 12601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:17.966178 12601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:17.966296 12601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:18.004576 12601 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:40337
I20260810 14:41:18.004594 12792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:40337 every 8 connection(s)
I20260810 14:41:18.005940 12601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:41:18.015089 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 12601
I20260810 14:41:18.015328 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:41:18.016112 21725 external_mini_cluster.cc:1428] Setting key 4d3a2aa9b6092ffa54ecc9f76ab856b4
Aug 10 14:41:18 dist-test-slave-h6n0 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372877, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:18.029838 12797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.008692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:54457 (local address 127.21.55.126:38449)
0810 14:41:18.009207 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:18.009213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:18.010139 (+ 926us) server_negotiation.cc:407] Connection header received
0810 14:41:18.012176 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:18.012187 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:18.012442 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:18.012555 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:18.014051 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.015109 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.016948 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.017492 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.017647 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.022067 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:18.022072 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:18.022076 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:18.022110 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:18.026167 (+ 4057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:18.027078 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:18.027085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:18.027089 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:18.027218 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:18.027702 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:18.027706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:18.027708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:18.027989 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:18.028168 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:18.028928 (+ 760us) server_negotiation.cc:299] Negotiation successful
0810 14:41:18.029444 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":239,"threads_started":1}
I20260810 14:41:18.030956 12798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.009227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:54457)
0810 14:41:18.009987 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:18.010026 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:18.011757 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:18.012771 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:18.012779 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:18.013309 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:18.013860 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.013877 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.015281 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.015292 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.016776 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.016797 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.017784 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.017791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.018078 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.018912 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:18.018936 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:18.021795 (+ 2859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:18.026337 (+ 4542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:18.026347 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:18.026364 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:18.026937 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:18.027347 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:18.027353 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:18.027357 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:18.027556 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:18.028158 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:18.028166 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:18.028513 (+ 347us) client_negotiation.cc:769] Sending connection context
0810 14:41:18.028814 (+ 301us) client_negotiation.cc:240] Negotiation successful
0810 14:41:18.029686 (+ 872us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":312,"threads_started":1}
I20260810 14:41:18.032707 12793 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38449
I20260810 14:41:18.033102 12793 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:18.034060 12793 heartbeater.cc:511] Master 127.21.55.126:38449 requested a full tablet report, sending...
I20260810 14:41:18.036338 12298 ts_manager.cc:194] Registered new tserver with Master: c833c6dff13c470a8a4e16a88062202f (127.21.55.66:40337)
I20260810 14:41:18.037668 12298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:54457
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:41:18.087967 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38449
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:41:18.449437 12595 heartbeater.cc:503] Master 127.21.55.126:38449 was elected leader, sending a full tablet report...
W20260810 14:41:18.499926 12801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:18.500383 12801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:18.500566 12801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:18.512143 12801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:18.512270 12801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:18.512456 12801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:41:18.524267 12801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:18.526273 12801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:18.528327 12801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:18.543794 12815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:18.544068 12820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:18.544425 12816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:18.546748 12801 server_base.cc:1034] running on GCE node
I20260810 14:41:18.547844 12801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:18.549604 12801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:18.550966 12801 hybrid_clock.cc:648] HybridClock initialized: now 1786372878550870 us; error 78 us; skew 500 ppm
I20260810 14:41:18.551535 12801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:18 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372878, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:18.555778 12801 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:41:18.557593 12801 webserver.cc:460] Webserver started at http://127.21.55.67:38071/ using document root <none> and password file <none>
I20260810 14:41:18.558354 12801 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:18.558575 12801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:18.558910 12801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:18.561932 12801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "9a09ae3be792404dbfe7df68e7db7305"
format_stamp: "Formatted at 2026-08-10 14:41:18 on dist-test-slave-h6n0"
server_key: "5ac5c607c761c46eef096a6d15e960db"
server_key_iv: "f61a722f0f07c31ad2ea447519961038"
server_key_version: "encryptionkey@0"
I20260810 14:41:18.562855 12801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "9a09ae3be792404dbfe7df68e7db7305"
format_stamp: "Formatted at 2026-08-10 14:41:18 on dist-test-slave-h6n0"
server_key: "5ac5c607c761c46eef096a6d15e960db"
server_key_iv: "f61a722f0f07c31ad2ea447519961038"
server_key_version: "encryptionkey@0"
I20260810 14:41:18.568630 12801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 14:41:18.572417 12831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:18.574620 12801 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 14:41:18.574854 12801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "9a09ae3be792404dbfe7df68e7db7305"
format_stamp: "Formatted at 2026-08-10 14:41:18 on dist-test-slave-h6n0"
server_key: "5ac5c607c761c46eef096a6d15e960db"
server_key_iv: "f61a722f0f07c31ad2ea447519961038"
server_key_version: "encryptionkey@0"
I20260810 14:41:18.575115 12801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:18.592090 12801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:18.597986 12801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:18.598272 12801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:18.599282 12801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:18.600713 12801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:18.600792 12801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:18.600880 12801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:18.600927 12801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:18.642982 12801 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:34635
I20260810 14:41:18.642994 13003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:34635 every 8 connection(s)
I20260810 14:41:18.644397 12801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:41:18.654440 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 12801
I20260810 14:41:18.654721 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:41:18.655356 21725 external_mini_cluster.cc:1428] Setting key 70efec2ded4bee44c52340473fc34af1
Aug 10 14:41:18 dist-test-slave-h6n0 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372878, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:18.669772 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.648288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:44567 (local address 127.21.55.126:38449)
0810 14:41:18.649014 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:18.649054 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:18.650674 (+ 1620us) server_negotiation.cc:407] Connection header received
0810 14:41:18.652779 (+ 2105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:18.652786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:18.652896 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:18.653034 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:18.655168 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.656034 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.658437 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.659165 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.659289 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.662956 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:18.662962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:18.662968 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:18.663006 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:18.666704 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:18.667499 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:18.667503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:18.667505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:18.667645 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:18.668090 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:18.668093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:18.668095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:18.668365 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:18.668478 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:18.669300 (+ 822us) server_negotiation.cc:299] Negotiation successful
0810 14:41:18.669478 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":365,"threads_started":1}
I20260810 14:41:18.671092 13008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.649403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:44567)
0810 14:41:18.650411 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:18.650457 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:18.652453 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:18.653310 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:18.653320 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:18.653951 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:18.654944 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.654973 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.656216 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.656221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.658258 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.658275 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.659442 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.659453 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.659693 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.660427 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:18.660452 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:18.662735 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:18.666889 (+ 4154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:18.666907 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:18.666919 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:18.667346 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:18.667746 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:18.667750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:18.667752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:18.667954 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:18.668507 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:18.668516 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:18.668925 (+ 409us) client_negotiation.cc:769] Sending connection context
0810 14:41:18.669161 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 14:41:18.669581 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":411,"threads_started":1}
I20260810 14:41:18.672816 13004 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38449
I20260810 14:41:18.673326 13004 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:18.674288 13004 heartbeater.cc:511] Master 127.21.55.126:38449 requested a full tablet report, sending...
I20260810 14:41:18.676326 12298 ts_manager.cc:194] Registered new tserver with Master: 9a09ae3be792404dbfe7df68e7db7305 (127.21.55.67:34635)
I20260810 14:41:18.677336 12298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:44567
I20260810 14:41:18.685081 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:41:18.707643 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.691674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.21.55.126:38449)
0810 14:41:18.691931 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:18.691943 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:18.692504 (+ 561us) server_negotiation.cc:407] Connection header received
0810 14:41:18.692797 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:18.692804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:18.692941 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:18.693075 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:18.695029 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.695785 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.697316 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.697764 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.697917 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.701693 (+ 3776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:18.701699 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:18.701702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:18.701736 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:18.704811 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:18.705528 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:18.705541 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:18.705544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:18.705662 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:18.706067 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:18.706073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:18.706078 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:18.706421 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:18.706702 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:18.707076 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 14:41:18.707310 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:41:18.712172 12298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:41:18.719372 12298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:41:18.761171 13033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.734462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.21.55.67:34635)
0810 14:41:18.747302 (+ 12840us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:18.747316 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:18.750933 (+ 3617us) server_negotiation.cc:407] Connection header received
0810 14:41:18.751055 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:18.751064 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:18.751288 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:18.751507 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:18.752822 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.754505 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.758669 (+ 4164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.759801 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.759970 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.760253 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:18.760562 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 14:41:18.760744 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12572,"thread_start_us":12362,"threads_started":1}
I20260810 14:41:18.761258 13029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.732898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:34635 (local address 127.0.0.1:46058)
0810 14:41:18.750612 (+ 17714us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:18.750634 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:18.750782 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:18.751818 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:18.751822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:18.751849 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:18.752425 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.752451 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.754757 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.754764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.758415 (+ 3651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.758435 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.760044 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.760051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.760270 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.760332 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 14:41:18.760501 (+ 169us) client_negotiation.cc:240] Negotiation successful
0810 14:41:18.760753 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16265,"mutex_wait_us":269,"spinlock_wait_cycles":3072,"thread_start_us":14529,"threads_started":1}
I20260810 14:41:18.761868 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.732997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49714 (local address 127.21.55.65:45615)
0810 14:41:18.750243 (+ 17246us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:18.750252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:18.750290 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:41:18.750427 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:18.750440 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:18.750803 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:18.751881 (+ 1078us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:18.752169 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.754497 (+ 2328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.758979 (+ 4482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.760489 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.761044 (+ 555us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.761281 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:18.761357 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 14:41:18.761526 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17053,"thread_start_us":16843,"threads_started":1}
I20260810 14:41:18.761914 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.732601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45615 (local address 127.0.0.1:49714)
0810 14:41:18.748168 (+ 15567us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:18.748279 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:18.748534 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:18.751125 (+ 2591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:18.751133 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:18.751176 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:18.751667 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.751682 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.754791 (+ 3109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.754799 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.757977 (+ 3178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.757998 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.760759 (+ 2761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.760765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.760939 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.761009 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 14:41:18.761113 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 14:41:18.761454 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15202,"mutex_wait_us":16,"thread_start_us":16366,"threads_started":1}
I20260810 14:41:18.762718 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.749704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:40337 (local address 127.0.0.1:57344)
0810 14:41:18.750612 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:18.750634 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:18.750782 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:18.751989 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:18.752000 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:18.752020 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:18.753112 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.753149 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.755501 (+ 2352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.755509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.758472 (+ 2963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.758486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.760724 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.760733 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.760933 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.762119 (+ 1186us) client_negotiation.cc:769] Sending connection context
0810 14:41:18.762216 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 14:41:18.762338 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":188,"spinlock_wait_cycles":11136,"thread_start_us":585,"threads_started":1}
I20260810 14:41:18.765798 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.750104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57344 (local address 127.21.55.66:40337)
0810 14:41:18.751411 (+ 1307us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:18.751420 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:18.751447 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:41:18.751551 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:18.751560 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:18.751761 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:18.752340 (+ 579us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:18.753551 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.755292 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.758658 (+ 3366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.760209 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.760363 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.760539 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:18.762693 (+ 2154us) server_negotiation.cc:299] Negotiation successful
0810 14:41:18.765450 (+ 2757us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1050,"thread_start_us":1697,"threads_started":1}
I20260810 14:41:18.765327 12909 tablet_service.cc:1467] Processing CreateTablet for tablet bf96afde3a2c479898ff38d43b5ea8d4 (DEFAULT_TABLE table=test-table [id=77f2d44e0e1f4957b165abccbf7f41fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:18.766364 12494 tablet_service.cc:1467] Processing CreateTablet for tablet bf96afde3a2c479898ff38d43b5ea8d4 (DEFAULT_TABLE table=test-table [id=77f2d44e0e1f4957b165abccbf7f41fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:18.767935 12909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf96afde3a2c479898ff38d43b5ea8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:18.768288 12494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf96afde3a2c479898ff38d43b5ea8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:18.769662 12696 tablet_service.cc:1467] Processing CreateTablet for tablet bf96afde3a2c479898ff38d43b5ea8d4 (DEFAULT_TABLE table=test-table [id=77f2d44e0e1f4957b165abccbf7f41fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:18.771036 12696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf96afde3a2c479898ff38d43b5ea8d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:18.779990 13039 tablet_bootstrap.cc:492] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: Bootstrap starting.
I20260810 14:41:18.782600 13040 tablet_bootstrap.cc:492] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: Bootstrap starting.
I20260810 14:41:18.784610 13039 tablet_bootstrap.cc:654] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:18.785219 13041 tablet_bootstrap.cc:492] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f: Bootstrap starting.
I20260810 14:41:18.787398 13039 log.cc:824] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:18.787787 13040 tablet_bootstrap.cc:654] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:18.789034 13041 tablet_bootstrap.cc:654] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:18.789322 13040 log.cc:824] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:18.790396 13041 log.cc:824] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:18.792033 13039 tablet_bootstrap.cc:492] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: No bootstrap required, opened a new log
I20260810 14:41:18.792387 13039 ts_tablet_manager.cc:1397] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260810 14:41:18.793085 13040 tablet_bootstrap.cc:492] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: No bootstrap required, opened a new log
I20260810 14:41:18.793186 13041 tablet_bootstrap.cc:492] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f: No bootstrap required, opened a new log
I20260810 14:41:18.793350 13040 ts_tablet_manager.cc:1397] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260810 14:41:18.793449 13041 ts_tablet_manager.cc:1397] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 14:41:18.800905 13041 raft_consensus.cc:348] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:18.801290 13041 raft_consensus.cc:374] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:18.801403 13041 raft_consensus.cc:729] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c833c6dff13c470a8a4e16a88062202f, State: Initialized, Role: FOLLOWER
I20260810 14:41:18.802027 13041 consensus_queue.cc:260] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:18.803377 13039 raft_consensus.cc:348] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:18.803795 13039 raft_consensus.cc:374] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:18.803920 13039 raft_consensus.cc:729] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3aaa29a27343b6b9a6ab82892f22fe, State: Initialized, Role: FOLLOWER
I20260810 14:41:18.804071 13040 raft_consensus.cc:348] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:18.804525 13040 raft_consensus.cc:374] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:18.804633 13040 raft_consensus.cc:729] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a09ae3be792404dbfe7df68e7db7305, State: Initialized, Role: FOLLOWER
I20260810 14:41:18.804683 12793 heartbeater.cc:503] Master 127.21.55.126:38449 was elected leader, sending a full tablet report...
I20260810 14:41:18.804628 13039 consensus_queue.cc:260] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:18.805330 13041 ts_tablet_manager.cc:1428] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
I20260810 14:41:18.805331 13040 consensus_queue.cc:260] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:18.806865 13004 heartbeater.cc:503] Master 127.21.55.126:38449 was elected leader, sending a full tablet report...
I20260810 14:41:18.807098 13039 ts_tablet_manager.cc:1428] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260810 14:41:18.808498 13040 ts_tablet_manager.cc:1428] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260810 14:41:18.881283 13046 raft_consensus.cc:482] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:41:18.881675 13046 raft_consensus.cc:504] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:18.883754 13046 leader_election.cc:288] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c833c6dff13c470a8a4e16a88062202f (127.21.55.66:40337), 9a09ae3be792404dbfe7df68e7db7305 (127.21.55.67:34635)
I20260810 14:41:18.892480 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.884240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:39281 (local address 127.21.55.66:40337)
0810 14:41:18.884495 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:18.884502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:18.885135 (+ 633us) server_negotiation.cc:407] Connection header received
0810 14:41:18.885406 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:18.885411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:18.885479 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:18.885590 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:18.886444 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.887651 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.889955 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.891297 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.891534 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.891622 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:18.892037 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 14:41:18.892202 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 14:41:18.892535 13048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.883961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:40337 (local address 127.21.55.65:39281)
0810 14:41:18.885005 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:18.885023 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:18.885222 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:18.885616 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:18.885620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:18.885650 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:18.886020 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.886030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.887885 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.887893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.889698 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.889719 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.891567 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.891573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.891798 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.891837 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 14:41:18.891951 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 14:41:18.892146 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":708,"threads_started":1}
I20260810 14:41:18.893014 13033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.884282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:47203 (local address 127.21.55.67:34635)
0810 14:41:18.884495 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:18.884502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:18.885674 (+ 1172us) server_negotiation.cc:407] Connection header received
0810 14:41:18.885890 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:18.885895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:18.885967 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:18.886064 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:18.886789 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.887654 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.889746 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.891422 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.891614 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.891702 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:18.892571 (+ 869us) server_negotiation.cc:299] Negotiation successful
0810 14:41:18.892756 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:41:18.893615 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.884536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:34635 (local address 127.21.55.65:47203)
0810 14:41:18.885562 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:18.885577 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:18.885739 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:18.886080 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:18.886084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:18.886107 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:18.886625 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.886641 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.887789 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.887796 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.889570 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:18.889584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.891796 (+ 2212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.891802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:18.892346 (+ 544us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.892382 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 14:41:18.893197 (+ 815us) client_negotiation.cc:240] Negotiation successful
0810 14:41:18.893348 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"mutex_wait_us":4,"thread_start_us":675,"threads_started":1}
I20260810 14:41:18.894011 12725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf96afde3a2c479898ff38d43b5ea8d4" candidate_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c833c6dff13c470a8a4e16a88062202f" is_pre_election: true
I20260810 14:41:18.894667 12725 raft_consensus.cc:2454] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe3aaa29a27343b6b9a6ab82892f22fe in term 0.
I20260810 14:41:18.894704 12935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf96afde3a2c479898ff38d43b5ea8d4" candidate_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a09ae3be792404dbfe7df68e7db7305" is_pre_election: true
I20260810 14:41:18.895329 12935 raft_consensus.cc:2454] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe3aaa29a27343b6b9a6ab82892f22fe in term 0.
I20260810 14:41:18.895448 12422 leader_election.cc:302] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c833c6dff13c470a8a4e16a88062202f, fe3aaa29a27343b6b9a6ab82892f22fe; no voters:
I20260810 14:41:18.896052 13046 raft_consensus.cc:2781] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:41:18.896227 13046 raft_consensus.cc:482] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:41:18.896342 13046 raft_consensus.cc:3037] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Advancing to term 1
W20260810 14:41:18.896844 13005 tablet.cc:2369] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:41:18.899297 13046 raft_consensus.cc:504] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:18.900003 13046 leader_election.cc:288] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [CANDIDATE]: Term 1 election: Requested vote from peers c833c6dff13c470a8a4e16a88062202f (127.21.55.66:40337), 9a09ae3be792404dbfe7df68e7db7305 (127.21.55.67:34635)
I20260810 14:41:18.900885 12725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf96afde3a2c479898ff38d43b5ea8d4" candidate_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c833c6dff13c470a8a4e16a88062202f"
I20260810 14:41:18.901145 12935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf96afde3a2c479898ff38d43b5ea8d4" candidate_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a09ae3be792404dbfe7df68e7db7305"
I20260810 14:41:18.901397 12935 raft_consensus.cc:3037] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:18.901460 12725 raft_consensus.cc:3037] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:18.904117 12935 raft_consensus.cc:2454] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe3aaa29a27343b6b9a6ab82892f22fe in term 1.
I20260810 14:41:18.904598 12424 leader_election.cc:302] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a09ae3be792404dbfe7df68e7db7305, fe3aaa29a27343b6b9a6ab82892f22fe; no voters:
I20260810 14:41:18.904954 13046 raft_consensus.cc:2781] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:41:18.905485 13046 raft_consensus.cc:686] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 1 LEADER]: Becoming Leader. State: Replica: fe3aaa29a27343b6b9a6ab82892f22fe, State: Running, Role: LEADER
I20260810 14:41:18.905520 12725 raft_consensus.cc:2454] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe3aaa29a27343b6b9a6ab82892f22fe in term 1.
I20260810 14:41:18.906091 13046 consensus_queue.cc:237] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:18.912055 12295 catalog_manager.cc:5495] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe reported cstate change: term changed from 0 to 1, leader changed from <none> to fe3aaa29a27343b6b9a6ab82892f22fe (127.21.55.65). New cstate: current_term: 1 leader_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } health_report { overall_health: UNKNOWN } } }
W20260810 14:41:18.913442 12596 tablet.cc:2369] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:41:18.945923 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.939626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.21.55.65:45615)
0810 14:41:18.939945 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:18.939951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:18.939980 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:41:18.940088 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:18.940092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:18.940199 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:18.940349 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:41:18.940820 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.941816 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.943451 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:18.943932 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:18.944059 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:18.944616 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:41:18.945202 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:41:18.945322 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:18.945460 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 14:41:18.945531 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260810 14:41:18.955624 12935 raft_consensus.cc:1261] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305 [term 1 FOLLOWER]: Refusing update from remote peer fe3aaa29a27343b6b9a6ab82892f22fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:41:18.955626 12725 raft_consensus.cc:1261] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f [term 1 FOLLOWER]: Refusing update from remote peer fe3aaa29a27343b6b9a6ab82892f22fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:41:18.956715 13051 consensus_queue.cc:1035] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:41:18.957216 13046 consensus_queue.cc:1035] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:41:18.988317 12297 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:41:18.988708 12297 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:41:18.993098 12297 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Sending DeleteTablet for 3 replicas of tablet bf96afde3a2c479898ff38d43b5ea8d4
I20260810 14:41:18.994472 12909 tablet_service.cc:1514] Processing DeleteTablet for tablet bf96afde3a2c479898ff38d43b5ea8d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:18 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:46058
I20260810 14:41:18.994755 12494 tablet_service.cc:1514] Processing DeleteTablet for tablet bf96afde3a2c479898ff38d43b5ea8d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:18 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:49714
I20260810 14:41:18.994709 12696 tablet_service.cc:1514] Processing DeleteTablet for tablet bf96afde3a2c479898ff38d43b5ea8d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:18 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:57344
I20260810 14:41:18.997188 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 12395
I20260810 14:41:19.010057 13073 tablet_replica.cc:330] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f: stopping tablet replica
I20260810 14:41:19.010520 13074 tablet_replica.cc:330] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: stopping tablet replica
I20260810 14:41:19.010998 13074 raft_consensus.cc:2229] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:41:19.011608 13074 raft_consensus.cc:2258] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:19.012247 13073 raft_consensus.cc:2229] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:41:19.012704 13073 raft_consensus.cc:2258] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:19.014832 13073 ts_tablet_manager.cc:1905] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:19.016016 13074 ts_tablet_manager.cc:1905] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:19.023159 13073 ts_tablet_manager.cc:1918] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:41:19.023695 13073 log.cc:1196] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/bf96afde3a2c479898ff38d43b5ea8d4
I20260810 14:41:19.024645 13073 ts_tablet_manager.cc:1939] T bf96afde3a2c479898ff38d43b5ea8d4 P c833c6dff13c470a8a4e16a88062202f: Deleting consensus metadata
I20260810 14:41:19.024662 13072 tablet_replica.cc:330] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: stopping tablet replica
I20260810 14:41:19.025152 13072 raft_consensus.cc:2229] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:41:19.026671 12274 catalog_manager.cc:4841] TS c833c6dff13c470a8a4e16a88062202f (127.21.55.66:40337): tablet bf96afde3a2c479898ff38d43b5ea8d4 (table test-table [id=77f2d44e0e1f4957b165abccbf7f41fd]) successfully deleted
I20260810 14:41:19.029826 13072 raft_consensus.cc:2258] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:19.032557 13072 ts_tablet_manager.cc:1905] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 14:41:19 dist-test-slave-h6n0 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:41:19.036381 13074 ts_tablet_manager.cc:1918] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:41:19.040721 13072 ts_tablet_manager.cc:1918] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:41:19.040984 13072 log.cc:1196] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/bf96afde3a2c479898ff38d43b5ea8d4
I20260810 14:41:19.041363 13074 log.cc:1196] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/bf96afde3a2c479898ff38d43b5ea8d4
I20260810 14:41:19.041608 13072 ts_tablet_manager.cc:1939] T bf96afde3a2c479898ff38d43b5ea8d4 P 9a09ae3be792404dbfe7df68e7db7305: Deleting consensus metadata
I20260810 14:41:19.042192 13074 ts_tablet_manager.cc:1939] T bf96afde3a2c479898ff38d43b5ea8d4 P fe3aaa29a27343b6b9a6ab82892f22fe: Deleting consensus metadata
I20260810 14:41:19.043919 12279 catalog_manager.cc:4841] TS 9a09ae3be792404dbfe7df68e7db7305 (127.21.55.67:34635): tablet bf96afde3a2c479898ff38d43b5ea8d4 (table test-table [id=77f2d44e0e1f4957b165abccbf7f41fd]) successfully deleted
I20260810 14:41:19.044266 12274 catalog_manager.cc:4841] TS fe3aaa29a27343b6b9a6ab82892f22fe (127.21.55.65:45615): tablet bf96afde3a2c479898ff38d43b5ea8d4 (table test-table [id=77f2d44e0e1f4957b165abccbf7f41fd]) successfully deleted
I20260810 14:41:19.046224 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:18.998453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.21.55.65:45615)
0810 14:41:18.998691 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:18.998698 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:19.021000 (+ 22302us) server_negotiation.cc:407] Connection header received
0810 14:41:19.021588 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:19.021595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:19.021691 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:19.021789 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:19.025527 (+ 3738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:19.026930 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:19.029303 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:19.030004 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:19.030125 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:19.036126 (+ 6001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:19.036144 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:19.036164 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:19.036221 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:19.040941 (+ 4720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:19.042402 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:19.042415 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:19.042423 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:19.042670 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:19.043251 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:19.043257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:19.043261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:19.043782 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:19.044133 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:19.045223 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0810 14:41:19.045601 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260810 14:41:19.046970 12585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:19.154557 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 12395
I20260810 14:41:19.181911 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 12601
Aug 10 14:41:19 dist-test-slave-h6n0 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:41:19.198457 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:19.182848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43814 (local address 127.21.55.66:40337)
0810 14:41:19.183061 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:19.183066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:19.183087 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:41:19.183153 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:19.183157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:19.183225 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:19.183329 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:19.184718 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:19.185873 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:19.187481 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:19.187897 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:19.188003 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:19.192580 (+ 4577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:19.192593 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:19.192610 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:19.192642 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:19.195599 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:19.196312 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:19.196319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:19.196321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:19.196450 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:19.196862 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:19.196868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:19.196870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:19.197246 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:19.197577 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:19.197869 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 14:41:19.198093 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:41:19.199304 12785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:19.300928 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 12601
I20260810 14:41:19.330027 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 12801
Aug 10 14:41:19 dist-test-slave-h6n0 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:41:19.346902 13033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:19.331059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.21.55.67:34635)
0810 14:41:19.331263 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:19.331269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:19.331294 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:41:19.331368 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:19.331373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:19.331450 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:19.331574 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:19.332879 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:19.333585 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:19.335061 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:19.335452 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:19.335592 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:19.340572 (+ 4980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:19.340589 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:19.340606 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:19.340641 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:19.343693 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:19.344481 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:19.344489 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:19.344491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:19.344629 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:19.345356 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:19.345360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:19.345363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:19.345734 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:19.346035 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:19.346335 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 14:41:19.346516 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:41:19.347651 12995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:19.443001 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 12801
I20260810 14:41:19.472101 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 12249
I20260810 14:41:19.473433 12346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:19.590198 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 12249
I20260810 14:41:19.614264 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38449
--webserver_interface=127.21.55.126
--webserver_port=45413
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:38449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:19.981909 13085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:19.982187 13085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:19.982314 13085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:19.989706 13085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:41:19.989812 13085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:19.989923 13085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:19.990023 13085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:41:19.990093 13085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:41:19.999977 13085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:38449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=45413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:20.001506 13085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:20.003443 13085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:20.029064 13092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:20.028980 13094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:20.029064 13101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:20.029891 13085 server_base.cc:1034] running on GCE node
I20260810 14:41:20.030929 13085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:20.032634 13085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:20.034067 13085 hybrid_clock.cc:648] HybridClock initialized: now 1786372880033935 us; error 53 us; skew 500 ppm
I20260810 14:41:20.035007 13085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:20 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372880, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:20.039422 13085 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:41:20.041453 13085 webserver.cc:460] Webserver started at http://127.21.55.126:45413/ using document root <none> and password file <none>
I20260810 14:41:20.042286 13085 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:20.042464 13085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:20.049690 13085 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:41:20.053544 13110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:20.055503 13085 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:41:20.055745 13085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "d0eabdf2f3bb49d3991048f5981a7882"
format_stamp: "Formatted at 2026-08-10 14:41:16 on dist-test-slave-h6n0"
server_key: "afa95c2cbd4349e855aedc7a7ca75d9c"
server_key_iv: "c67b546f0ba3aeb73c8b81bd4036e037"
server_key_version: "encryptionkey@0"
I20260810 14:41:20.056993 13085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:20.096452 13085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:20.101181 13085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:20.101477 13085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:20.122309 13085 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:38449
I20260810 14:41:20.122323 13186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:38449 every 8 connection(s)
I20260810 14:41:20.123988 13085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:41:20.125406 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 13085
I20260810 14:41:20.126247 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:45615
--local_ip_for_outbound_sockets=127.21.55.65
--tserver_master_addrs=127.21.55.126:38449
--webserver_port=36389
--webserver_interface=127.21.55.65
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:41:20.132937 13187 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:41:20.137468 13187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Bootstrap starting.
I20260810 14:41:20.158951 13187 log.cc:824] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:20.173952 13187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:41:20.174690 13187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Bootstrap complete.
I20260810 14:41:20.184451 13187 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:20.185599 13187 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0eabdf2f3bb49d3991048f5981a7882, State: Initialized, Role: FOLLOWER
I20260810 14:41:20.186372 13187 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:20.186720 13187 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:41:20.186852 13187 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:41:20.187042 13187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:41:20.191288 13187 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:20.191856 13187 leader_election.cc:302] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0eabdf2f3bb49d3991048f5981a7882; no voters:
I20260810 14:41:20.192633 13187 leader_election.cc:288] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:41:20.192832 13196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:41:20.194059 13196 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 2 LEADER]: Becoming Leader. State: Replica: d0eabdf2f3bb49d3991048f5981a7882, State: Running, Role: LEADER
I20260810 14:41:20.194970 13187 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:41:20.194809 13196 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:20.198503 13199 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0eabdf2f3bb49d3991048f5981a7882. Latest consensus state: current_term: 2 leader_uuid: "d0eabdf2f3bb49d3991048f5981a7882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } } }
I20260810 14:41:20.199023 13199 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:20.198642 13197 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d0eabdf2f3bb49d3991048f5981a7882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } } }
I20260810 14:41:20.199579 13197 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:20.201546 13204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:41:20.212020 13204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:41:20.213788 13204 catalog_manager.cc:1259] Loaded cluster ID: 5106100e13c44ac88cfaec46a1f922b9
I20260810 14:41:20.213979 13204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:41:20.217954 13204 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 14:41:20.247491 13204 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 14:41:20.262372 13204 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:41:20.264292 13204 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Loaded TSK: 0
I20260810 14:41:20.265957 13204 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:41:20.537042 13191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:20.537339 13191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:20.537492 13191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:20.546396 13191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:20.546623 13191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:20.546826 13191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:41:20.558667 13191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:45615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=36389
--tserver_master_addrs=127.21.55.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:20.560153 13191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:20.561765 13191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:20.578377 13237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:20.578389 13235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:20.579739 13242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:20.580443 13191 server_base.cc:1034] running on GCE node
I20260810 14:41:20.581382 13191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:20.582731 13191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:20.584017 13191 hybrid_clock.cc:648] HybridClock initialized: now 1786372880583957 us; error 47 us; skew 500 ppm
I20260810 14:41:20.584381 13191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:20 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372880, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:20.588769 13191 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:41:20.590998 13191 webserver.cc:460] Webserver started at http://127.21.55.65:36389/ using document root <none> and password file <none>
I20260810 14:41:20.591761 13191 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:20.591933 13191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:20.598807 13191 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.004s
I20260810 14:41:20.601955 13257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:20.603677 13191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:41:20.603904 13191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "fe3aaa29a27343b6b9a6ab82892f22fe"
format_stamp: "Formatted at 2026-08-10 14:41:17 on dist-test-slave-h6n0"
server_key: "b16eb5c5f25cf51487f635ddea6ffafc"
server_key_iv: "6cd8fe03b09749364dc53e9231169e1d"
server_key_version: "encryptionkey@0"
I20260810 14:41:20.605355 13191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:20.622754 13191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:20.628229 13191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:20.628580 13191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:20.629777 13191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:20.631464 13191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:20.631623 13191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:20.631759 13191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:20.631858 13191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:20.672117 13191 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:45615
I20260810 14:41:20.672168 13427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:45615 every 8 connection(s)
I20260810 14:41:20.673583 13191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:41:20.679761 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 13191
I20260810 14:41:20.680486 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:40337
--local_ip_for_outbound_sockets=127.21.55.66
--tserver_master_addrs=127.21.55.126:38449
--webserver_port=33569
--webserver_interface=127.21.55.66
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:41:20 dist-test-slave-h6n0 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372880, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:20.706653 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:20.677702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:45855)
0810 14:41:20.678716 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:20.678772 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:20.680902 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:20.682383 (+ 1481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:20.682396 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:20.683135 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:20.683701 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:20.683720 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:20.685548 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:20.685553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:20.687296 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:20.687322 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:20.689856 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:20.689868 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:20.690350 (+ 482us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:20.691398 (+ 1048us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:20.691431 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:20.694225 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:20.700803 (+ 6578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:20.700811 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:20.700825 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:20.701343 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:20.702164 (+ 821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:20.702170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:20.702280 (+ 110us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:20.702716 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:20.704037 (+ 1321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:20.704051 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:20.704669 (+ 618us) client_negotiation.cc:769] Sending connection context
0810 14:41:20.704926 (+ 257us) client_negotiation.cc:240] Negotiation successful
0810 14:41:20.705519 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":427,"threads_started":1}
I20260810 14:41:20.707199 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:20.677387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:45855 (local address 127.21.55.126:38449)
0810 14:41:20.677983 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:20.677995 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:20.678953 (+ 958us) server_negotiation.cc:407] Connection header received
0810 14:41:20.681502 (+ 2549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:20.681515 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:20.681843 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:20.682233 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:20.683967 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:20.685331 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:20.687495 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:20.688777 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:20.689178 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:20.694745 (+ 5567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:20.694766 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:20.694813 (+ 47us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:20.694875 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:20.700567 (+ 5692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:20.701512 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:20.701527 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:20.701538 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:20.701820 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:20.702896 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:20.702905 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:20.702911 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:20.703713 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:20.703950 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:20.705287 (+ 1337us) server_negotiation.cc:299] Negotiation successful
0810 14:41:20.705811 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":266,"threads_started":1}
I20260810 14:41:20.710242 13430 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38449
I20260810 14:41:20.710687 13430 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:20.711860 13430 heartbeater.cc:511] Master 127.21.55.126:38449 requested a full tablet report, sending...
I20260810 14:41:20.714444 13137 ts_manager.cc:194] Registered new tserver with Master: fe3aaa29a27343b6b9a6ab82892f22fe (127.21.55.65:45615)
I20260810 14:41:20.715927 13137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:45855
W20260810 14:41:21.060636 13434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:21.060928 13434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:21.061237 13434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:21.069028 13434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:21.069151 13434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:21.069272 13434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:41:21.079895 13434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:40337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=33569
--tserver_master_addrs=127.21.55.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:21.081260 13434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:21.082641 13434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:21.097596 13447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:21.097680 13444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:21.097532 13449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:21.098639 13434 server_base.cc:1034] running on GCE node
I20260810 14:41:21.099591 13434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:21.100991 13434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:21.102308 13434 hybrid_clock.cc:648] HybridClock initialized: now 1786372881102190 us; error 105 us; skew 500 ppm
I20260810 14:41:21.102818 13434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:21 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372881, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:21.107124 13434 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:41:21.108958 13434 webserver.cc:460] Webserver started at http://127.21.55.66:33569/ using document root <none> and password file <none>
I20260810 14:41:21.109738 13434 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:21.109936 13434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:21.116037 13434 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:41:21.119391 13458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:21.120895 13434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:41:21.121140 13434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "c833c6dff13c470a8a4e16a88062202f"
format_stamp: "Formatted at 2026-08-10 14:41:17 on dist-test-slave-h6n0"
server_key: "671000839c2305d07ec6e3dd40927c9e"
server_key_iv: "d1fcdfd0759c7f95519f18a60efe2cf2"
server_key_version: "encryptionkey@0"
I20260810 14:41:21.122294 13434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:21.141503 13434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:21.146749 13434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:21.147063 13434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:21.148468 13434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:21.150123 13434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:21.150266 13434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:21.150437 13434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:21.150588 13434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:21.190402 13434 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:40337
I20260810 14:41:21.190495 13633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:40337 every 8 connection(s)
I20260810 14:41:21.191943 13434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:41:21.200269 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 13434
I20260810 14:41:21.200937 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:34635
--local_ip_for_outbound_sockets=127.21.55.67
--tserver_master_addrs=127.21.55.126:38449
--webserver_port=38071
--webserver_interface=127.21.55.67
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:41:21 dist-test-slave-h6n0 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372881, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:21.217399 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.194754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:45637 (local address 127.21.55.126:38449)
0810 14:41:21.194966 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:21.194986 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:21.196316 (+ 1330us) server_negotiation.cc:407] Connection header received
0810 14:41:21.198494 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:21.198500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:21.198824 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:21.199063 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:21.201515 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.202336 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.204380 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.204944 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.205114 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.209799 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:21.209805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:21.209808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:21.209878 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:21.213748 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:21.214578 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:21.214586 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:21.214592 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:21.214756 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:21.215241 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:21.215249 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:21.215255 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:21.215709 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:21.215859 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:21.216648 (+ 789us) server_negotiation.cc:299] Negotiation successful
0810 14:41:21.216930 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:41:21.218171 13638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.195337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:45637)
0810 14:41:21.196152 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:21.196198 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:21.198191 (+ 1993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:21.199538 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:21.199550 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:21.200301 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:21.201247 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.201275 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.202622 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.202628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.204173 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.204189 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.205246 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.205253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.205569 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.206625 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:21.206662 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:21.209486 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:21.213926 (+ 4440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:21.213933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:21.213948 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:21.214397 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:21.214876 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:21.214880 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:21.214882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:21.215095 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:21.215861 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:21.215868 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:21.216243 (+ 375us) client_negotiation.cc:769] Sending connection context
0810 14:41:21.216481 (+ 238us) client_negotiation.cc:240] Negotiation successful
0810 14:41:21.216909 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":282,"threads_started":1}
I20260810 14:41:21.219882 13635 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38449
I20260810 14:41:21.220222 13635 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:21.221201 13635 heartbeater.cc:511] Master 127.21.55.126:38449 requested a full tablet report, sending...
I20260810 14:41:21.223042 13137 ts_manager.cc:194] Registered new tserver with Master: c833c6dff13c470a8a4e16a88062202f (127.21.55.66:40337)
I20260810 14:41:21.224337 13137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:45637
W20260810 14:41:21.578944 13639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:21.579269 13639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:21.579370 13639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:21.588511 13639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:21.588636 13639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:21.588766 13639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:41:21.599982 13639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:34635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=38071
--tserver_master_addrs=127.21.55.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:21.601282 13639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:21.602572 13639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:21.619925 13649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:21.620108 13650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:21.620146 13654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:21.624361 13639 server_base.cc:1034] running on GCE node
I20260810 14:41:21.625113 13639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:21.626389 13639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:21.627676 13639 hybrid_clock.cc:648] HybridClock initialized: now 1786372881627599 us; error 66 us; skew 500 ppm
I20260810 14:41:21.628103 13639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:21 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372881, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:21.631788 13639 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:41:21.633390 13639 webserver.cc:460] Webserver started at http://127.21.55.67:38071/ using document root <none> and password file <none>
I20260810 14:41:21.634260 13639 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:21.634439 13639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:21.640285 13639 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:41:21.643389 13667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:21.644995 13639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:41:21.645203 13639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "9a09ae3be792404dbfe7df68e7db7305"
format_stamp: "Formatted at 2026-08-10 14:41:18 on dist-test-slave-h6n0"
server_key: "5ac5c607c761c46eef096a6d15e960db"
server_key_iv: "f61a722f0f07c31ad2ea447519961038"
server_key_version: "encryptionkey@0"
I20260810 14:41:21.646440 13639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:21.669342 13639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:21.674978 13639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:21.675294 13639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:21.676560 13639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:21.678285 13639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:21.678401 13639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:21.678515 13639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:21.678639 13639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260810 14:41:21.716882 13639 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:34635
I20260810 14:41:21.717006 13845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:34635 every 8 connection(s)
I20260810 14:41:21.718487 13639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:41:21.720098 13430 heartbeater.cc:503] Master 127.21.55.126:38449 was elected leader, sending a full tablet report...
I20260810 14:41:21.726280 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 13639
Aug 10 14:41:21 dist-test-slave-h6n0 krb5kdc[12230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372881, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:21.742484 13857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.727900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50394 (local address 127.21.55.126:38449)
0810 14:41:21.728884 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:21.728893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:21.728912 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:41:21.728963 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:21.728969 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:21.729040 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:21.729157 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:21.730749 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.731542 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.732952 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.733377 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.733525 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.736470 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:21.736475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:21.736478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:21.736523 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:21.739705 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:21.740404 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:21.740412 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:21.740415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:21.740573 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:21.740964 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:21.740976 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:21.740979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:21.741276 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:21.741556 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:21.741886 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 14:41:21.742129 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":542,"threads_started":1}
I20260810 14:41:21.743717 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.721813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:43099 (local address 127.21.55.126:38449)
0810 14:41:21.722410 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:21.722417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:21.723661 (+ 1244us) server_negotiation.cc:407] Connection header received
0810 14:41:21.725785 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:21.725791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:21.725895 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:21.726028 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:21.728362 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.729674 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.731519 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.731968 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.732130 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.736421 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:21.736427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:21.736430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:21.736468 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:21.739983 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:21.740784 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:21.740792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:21.740801 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:21.740926 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:21.741428 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:21.741432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:21.741434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:21.741703 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:21.741871 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:21.743084 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0810 14:41:21.743306 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":317,"threads_started":1}
I20260810 14:41:21.744709 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.722596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:43099)
0810 14:41:21.723476 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:21.723526 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:21.725469 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:21.726313 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:21.726322 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:21.727143 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:21.728159 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.728182 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.729852 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.729858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.731338 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.731352 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.732255 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.732264 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.732531 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.733328 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:21.733360 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:21.736138 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:21.740142 (+ 4004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:21.740152 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:21.740171 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:21.740636 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:21.741053 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:21.741060 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:21.741064 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:21.741276 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:21.741853 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:21.741861 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:21.742231 (+ 370us) client_negotiation.cc:769] Sending connection context
0810 14:41:21.742656 (+ 425us) client_negotiation.cc:240] Negotiation successful
0810 14:41:21.743275 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":355,"threads_started":1}
I20260810 14:41:21.746224 13847 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38449
I20260810 14:41:21.746639 13847 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:21.747450 13847 heartbeater.cc:511] Master 127.21.55.126:38449 requested a full tablet report, sending...
I20260810 14:41:21.748992 13137 ts_manager.cc:194] Registered new tserver with Master: 9a09ae3be792404dbfe7df68e7db7305 (127.21.55.67:34635)
I20260810 14:41:21.749851 13137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:43099
I20260810 14:41:21.758086 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:41:21.765838 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.759131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50404 (local address 127.21.55.126:38449)
0810 14:41:21.759303 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:21.759308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:21.759887 (+ 579us) server_negotiation.cc:407] Connection header received
0810 14:41:21.760099 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:21.760103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:21.760247 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:21.760376 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:41:21.760791 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.761595 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.763297 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.763791 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.763887 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.764695 (+ 808us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:41:21.765292 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:41:21.765445 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:21.765551 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 14:41:21.765621 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:41:21.767581 13137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:50404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:41:21.776285 13137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:41:21.802212 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.792377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43822 (local address 127.21.55.66:40337)
0810 14:41:21.793835 (+ 1458us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:21.793846 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:21.793878 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:41:21.793989 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:21.793998 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:21.794295 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:21.794509 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:21.795258 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.797005 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.799366 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.800867 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.801040 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.801272 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:21.801685 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 14:41:21.801855 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1288,"thread_start_us":1199,"threads_started":1}
I20260810 14:41:21.802313 13874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.792147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:40337 (local address 127.0.0.1:43822)
0810 14:41:21.792994 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:21.793051 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:21.793224 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:21.794554 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:21.794562 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:21.794592 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:21.795015 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.795029 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.797159 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.797164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.799189 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.799202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.801133 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.801139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.801429 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.801473 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 14:41:21.801628 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 14:41:21.801841 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":70,"thread_start_us":438,"threads_started":1}
I20260810 14:41:21.804653 13876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.793227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:34635 (local address 127.0.0.1:47842)
0810 14:41:21.794362 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:21.794378 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:21.794551 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:21.796790 (+ 2239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:21.796797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:21.796823 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:21.797235 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.797249 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.799411 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.799417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.801748 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.801763 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.803743 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.803750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.803936 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.803974 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 14:41:21.804087 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 14:41:21.804253 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1011,"mutex_wait_us":1,"thread_start_us":875,"threads_started":1}
I20260810 14:41:21.804905 13878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.794438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.21.55.67:34635)
0810 14:41:21.796069 (+ 1631us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:21.796077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:21.796109 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:41:21.796199 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:21.796205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:21.796438 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:21.797396 (+ 958us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:21.797622 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.799251 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.801972 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.803503 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.803644 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.803884 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:21.804297 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 14:41:21.804540 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1073,"thread_start_us":896,"threads_started":1}
I20260810 14:41:21.805246 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.793955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47428 (local address 127.21.55.65:45615)
0810 14:41:21.795160 (+ 1205us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:21.795170 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:21.796147 (+ 977us) server_negotiation.cc:407] Connection header received
0810 14:41:21.796442 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:21.796449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:21.796688 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:21.796874 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:21.797988 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.799532 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.802196 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.803693 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.803819 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.804069 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:21.804540 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 14:41:21.804712 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1016,"thread_start_us":821,"threads_started":1}
I20260810 14:41:21.806069 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.795218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45615 (local address 127.0.0.1:47428)
0810 14:41:21.796036 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:21.796053 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:21.796248 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:21.797153 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:21.797161 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:21.797187 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:21.797731 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.797748 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.799699 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.799705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.802007 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.802021 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.804068 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.804074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.804329 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.804361 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:41:21.805512 (+ 1151us) client_negotiation.cc:240] Negotiation successful
0810 14:41:21.805651 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":100,"thread_start_us":499,"threads_started":1}
I20260810 14:41:21.806840 13536 tablet_service.cc:1467] Processing CreateTablet for tablet f5b4eeb6db7f40938f3454d05bba2035 (DEFAULT_TABLE table=test-table [id=d9ad63686ce5448fa1dfdc25063dbfd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:21.808435 13536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5b4eeb6db7f40938f3454d05bba2035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:21.808270 13742 tablet_service.cc:1467] Processing CreateTablet for tablet f5b4eeb6db7f40938f3454d05bba2035 (DEFAULT_TABLE table=test-table [id=d9ad63686ce5448fa1dfdc25063dbfd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:21.809406 13742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5b4eeb6db7f40938f3454d05bba2035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:21.809554 13330 tablet_service.cc:1467] Processing CreateTablet for tablet f5b4eeb6db7f40938f3454d05bba2035 (DEFAULT_TABLE table=test-table [id=d9ad63686ce5448fa1dfdc25063dbfd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:21.810921 13330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5b4eeb6db7f40938f3454d05bba2035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:21.823419 13882 tablet_bootstrap.cc:492] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305: Bootstrap starting.
I20260810 14:41:21.825654 13883 tablet_bootstrap.cc:492] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: Bootstrap starting.
I20260810 14:41:21.827443 13882 tablet_bootstrap.cc:654] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:21.827993 13884 tablet_bootstrap.cc:492] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: Bootstrap starting.
I20260810 14:41:21.829365 13883 tablet_bootstrap.cc:654] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:21.830101 13882 log.cc:824] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:21.830674 13883 log.cc:824] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:21.831871 13884 tablet_bootstrap.cc:654] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:21.832800 13884 log.cc:824] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:21.833629 13883 tablet_bootstrap.cc:492] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: No bootstrap required, opened a new log
I20260810 14:41:21.833645 13882 tablet_bootstrap.cc:492] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305: No bootstrap required, opened a new log
I20260810 14:41:21.833914 13882 ts_tablet_manager.cc:1397] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.001s
I20260810 14:41:21.833914 13883 ts_tablet_manager.cc:1397] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 14:41:21.835371 13884 tablet_bootstrap.cc:492] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: No bootstrap required, opened a new log
I20260810 14:41:21.835654 13884 ts_tablet_manager.cc:1397] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 14:41:21.843014 13884 raft_consensus.cc:348] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:21.843406 13884 raft_consensus.cc:374] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:21.843502 13884 raft_consensus.cc:729] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3aaa29a27343b6b9a6ab82892f22fe, State: Initialized, Role: FOLLOWER
I20260810 14:41:21.844159 13884 consensus_queue.cc:260] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:21.844882 13882 raft_consensus.cc:348] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:21.845324 13882 raft_consensus.cc:374] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:21.845440 13882 raft_consensus.cc:729] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a09ae3be792404dbfe7df68e7db7305, State: Initialized, Role: FOLLOWER
I20260810 14:41:21.845204 13883 raft_consensus.cc:348] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:21.845630 13883 raft_consensus.cc:374] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:21.845738 13883 raft_consensus.cc:729] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c833c6dff13c470a8a4e16a88062202f, State: Initialized, Role: FOLLOWER
I20260810 14:41:21.846067 13884 ts_tablet_manager.cc:1428] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260810 14:41:21.846153 13882 consensus_queue.cc:260] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:21.846385 13883 consensus_queue.cc:260] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:21.847429 13847 heartbeater.cc:503] Master 127.21.55.126:38449 was elected leader, sending a full tablet report...
I20260810 14:41:21.847960 13635 heartbeater.cc:503] Master 127.21.55.126:38449 was elected leader, sending a full tablet report...
I20260810 14:41:21.848141 13882 ts_tablet_manager.cc:1428] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305: Time spent starting tablet: real 0.014s user 0.013s sys 0.003s
I20260810 14:41:21.849922 13883 ts_tablet_manager.cc:1428] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260810 14:41:21.890962 13896 raft_consensus.cc:482] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:41:21.891337 13896 raft_consensus.cc:504] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:21.893249 13896 leader_election.cc:288] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe3aaa29a27343b6b9a6ab82892f22fe (127.21.55.65:45615), 9a09ae3be792404dbfe7df68e7db7305 (127.21.55.67:34635)
I20260810 14:41:21.900539 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.893791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:57117 (local address 127.21.55.65:45615)
0810 14:41:21.894049 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:21.894059 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:21.894865 (+ 806us) server_negotiation.cc:407] Connection header received
0810 14:41:21.895091 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:21.895096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:21.895155 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:21.895257 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:21.895812 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.896659 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.898565 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.899469 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.899665 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.899743 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:21.900122 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 14:41:21.900278 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 14:41:21.900741 13878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.893987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:49623 (local address 127.21.55.67:34635)
0810 14:41:21.894282 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:21.894291 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:21.895050 (+ 759us) server_negotiation.cc:407] Connection header received
0810 14:41:21.895289 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:21.895296 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:21.895366 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:21.895463 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:21.896064 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.896843 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.898690 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.899745 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.899938 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.900012 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:21.900340 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 14:41:21.900468 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260810 14:41:21.900839 13900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.893561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45615 (local address 127.21.55.66:57117)
0810 14:41:21.894725 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:21.894744 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:21.894922 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:21.895275 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:21.895279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:21.895306 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:21.895664 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.895672 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.896790 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.896798 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.898371 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.898383 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.899701 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.899708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.899885 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.899921 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 14:41:21.900089 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 14:41:21.900229 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"mutex_wait_us":1,"thread_start_us":731,"threads_started":1}
I20260810 14:41:21.901415 13901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.893703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:34635 (local address 127.21.55.66:49623)
0810 14:41:21.894941 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:21.894956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:21.895101 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:21.895486 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:21.895490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:21.895515 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:21.895897 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.895930 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.897039 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.897044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.898485 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:21.898497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.899960 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.899967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:21.900154 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.900187 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:41:21.900914 (+ 727us) client_negotiation.cc:240] Negotiation successful
0810 14:41:21.901165 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"thread_start_us":855,"threads_started":1}
I20260810 14:41:21.901971 13360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b4eeb6db7f40938f3454d05bba2035" candidate_uuid: "c833c6dff13c470a8a4e16a88062202f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" is_pre_election: true
I20260810 14:41:21.902618 13360 raft_consensus.cc:2454] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c833c6dff13c470a8a4e16a88062202f in term 0.
I20260810 14:41:21.902475 13768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b4eeb6db7f40938f3454d05bba2035" candidate_uuid: "c833c6dff13c470a8a4e16a88062202f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a09ae3be792404dbfe7df68e7db7305" is_pre_election: true
I20260810 14:41:21.903017 13768 raft_consensus.cc:2454] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c833c6dff13c470a8a4e16a88062202f in term 0.
I20260810 14:41:21.903342 13461 leader_election.cc:302] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c833c6dff13c470a8a4e16a88062202f, fe3aaa29a27343b6b9a6ab82892f22fe; no voters:
I20260810 14:41:21.903836 13896 raft_consensus.cc:2781] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:41:21.903982 13896 raft_consensus.cc:482] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:41:21.904127 13896 raft_consensus.cc:3037] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:21.906491 13896 raft_consensus.cc:504] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:21.907158 13896 leader_election.cc:288] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [CANDIDATE]: Term 1 election: Requested vote from peers fe3aaa29a27343b6b9a6ab82892f22fe (127.21.55.65:45615), 9a09ae3be792404dbfe7df68e7db7305 (127.21.55.67:34635)
I20260810 14:41:21.907621 13768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b4eeb6db7f40938f3454d05bba2035" candidate_uuid: "c833c6dff13c470a8a4e16a88062202f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a09ae3be792404dbfe7df68e7db7305"
I20260810 14:41:21.907644 13360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5b4eeb6db7f40938f3454d05bba2035" candidate_uuid: "c833c6dff13c470a8a4e16a88062202f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe"
I20260810 14:41:21.907800 13768 raft_consensus.cc:3037] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:21.907872 13360 raft_consensus.cc:3037] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:21.910138 13768 raft_consensus.cc:2454] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c833c6dff13c470a8a4e16a88062202f in term 1.
I20260810 14:41:21.910264 13360 raft_consensus.cc:2454] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c833c6dff13c470a8a4e16a88062202f in term 1.
I20260810 14:41:21.910691 13464 leader_election.cc:302] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a09ae3be792404dbfe7df68e7db7305, c833c6dff13c470a8a4e16a88062202f; no voters:
I20260810 14:41:21.911240 13896 raft_consensus.cc:2781] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:41:21.911837 13896 raft_consensus.cc:686] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 1 LEADER]: Becoming Leader. State: Replica: c833c6dff13c470a8a4e16a88062202f, State: Running, Role: LEADER
I20260810 14:41:21.912322 13896 consensus_queue.cc:237] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } } peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } }
I20260810 14:41:21.916813 13135 catalog_manager.cc:5495] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f reported cstate change: term changed from 0 to 1, leader changed from <none> to c833c6dff13c470a8a4e16a88062202f (127.21.55.66). New cstate: current_term: 1 leader_uuid: "c833c6dff13c470a8a4e16a88062202f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c833c6dff13c470a8a4e16a88062202f" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 } health_report { overall_health: UNKNOWN } } }
W20260810 14:41:21.926014 13431 tablet.cc:2369] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:41:21.936772 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.931711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.21.55.66:40337)
0810 14:41:21.931897 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:21.931902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:21.931918 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:41:21.931998 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:21.932002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:21.932081 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:21.932207 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:41:21.932567 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.933286 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.934759 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.935169 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.935264 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:21.935724 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:41:21.936201 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:41:21.936308 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:21.936482 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 14:41:21.936570 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 14:41:21.944216 13636 tablet.cc:2369] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:41:21.946970 13360 raft_consensus.cc:1261] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe [term 1 FOLLOWER]: Refusing update from remote peer c833c6dff13c470a8a4e16a88062202f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:41:21.946966 13768 raft_consensus.cc:1261] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305 [term 1 FOLLOWER]: Refusing update from remote peer c833c6dff13c470a8a4e16a88062202f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:41:21.948508 13896 consensus_queue.cc:1035] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe3aaa29a27343b6b9a6ab82892f22fe" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 45615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:41:21.948925 13902 consensus_queue.cc:1035] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a09ae3be792404dbfe7df68e7db7305" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 34635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:41:21.978832 13136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:50404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:41:21.979166 13136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:50404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:41:21.982960 13136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Sending DeleteTablet for 3 replicas of tablet f5b4eeb6db7f40938f3454d05bba2035
I20260810 14:41:21.984373 13742 tablet_service.cc:1514] Processing DeleteTablet for tablet f5b4eeb6db7f40938f3454d05bba2035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:21 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:47842
I20260810 14:41:21.984510 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 13191
I20260810 14:41:21.984705 13536 tablet_service.cc:1514] Processing DeleteTablet for tablet f5b4eeb6db7f40938f3454d05bba2035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:21 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:43822
I20260810 14:41:21.985204 13330 tablet_service.cc:1514] Processing DeleteTablet for tablet f5b4eeb6db7f40938f3454d05bba2035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:21 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:47428
I20260810 14:41:21.997573 13918 tablet_replica.cc:330] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305: stopping tablet replica
I20260810 14:41:21.998245 13918 raft_consensus.cc:2229] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:41:21.998961 13918 raft_consensus.cc:2258] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:21.999837 13919 tablet_replica.cc:330] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: stopping tablet replica
I20260810 14:41:22.000360 13919 raft_consensus.cc:2229] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:41:22.001187 13919 raft_consensus.cc:2258] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:22.001614 13918 ts_tablet_manager.cc:1905] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:22.002146 13920 tablet_replica.cc:330] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: stopping tablet replica
I20260810 14:41:22.002604 13920 raft_consensus.cc:2229] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:41:22.003199 13920 raft_consensus.cc:2258] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:22.003957 13919 ts_tablet_manager.cc:1905] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:22.006373 13920 ts_tablet_manager.cc:1905] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:22.012151 13877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:21.985701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47444 (local address 127.21.55.65:45615)
0810 14:41:21.986298 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:21.986308 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:21.986333 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:41:21.986407 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:21.986417 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:21.986523 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:21.986685 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:21.989563 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.991412 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.994282 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:21.995478 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:21.995641 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:22.002915 (+ 7274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:22.002934 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:22.002953 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:22.003013 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:22.007408 (+ 4395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:22.008545 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:22.008561 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:22.008565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:22.009033 (+ 468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:22.009496 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:22.009502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:22.009507 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:22.010095 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:22.010418 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:22.011197 (+ 779us) server_negotiation.cc:299] Negotiation successful
0810 14:41:22.011713 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419}
I20260810 14:41:22.012938 13919 ts_tablet_manager.cc:1918] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:41:22.013176 13919 log.cc:1196] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/f5b4eeb6db7f40938f3454d05bba2035
I20260810 14:41:22.013263 13418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:22.013972 13919 ts_tablet_manager.cc:1939] T f5b4eeb6db7f40938f3454d05bba2035 P c833c6dff13c470a8a4e16a88062202f: Deleting consensus metadata
I20260810 14:41:22.015014 13918 ts_tablet_manager.cc:1918] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:41:22.015257 13918 log.cc:1196] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/f5b4eeb6db7f40938f3454d05bba2035
I20260810 14:41:22.015892 13918 ts_tablet_manager.cc:1939] T f5b4eeb6db7f40938f3454d05bba2035 P 9a09ae3be792404dbfe7df68e7db7305: Deleting consensus metadata
I20260810 14:41:22.016659 13114 catalog_manager.cc:4841] TS c833c6dff13c470a8a4e16a88062202f (127.21.55.66:40337): tablet f5b4eeb6db7f40938f3454d05bba2035 (table test-table [id=d9ad63686ce5448fa1dfdc25063dbfd4]) successfully deleted
I20260810 14:41:22.017802 13117 catalog_manager.cc:4841] TS 9a09ae3be792404dbfe7df68e7db7305 (127.21.55.67:34635): tablet f5b4eeb6db7f40938f3454d05bba2035 (table test-table [id=d9ad63686ce5448fa1dfdc25063dbfd4]) successfully deleted
I20260810 14:41:22.115408 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 13191
W20260810 14:41:22.149165 13114 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.55.65:45615 (error 108)
I20260810 14:41:22.149981 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 13434
I20260810 14:41:22.151448 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:22.150806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45615 (local address 127.0.0.1:47446)
0810 14:41:22.151094 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:22.151230 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:45615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
W20260810 14:41:22.151870 13114 catalog_manager.cc:4584] TS fe3aaa29a27343b6b9a6ab82892f22fe (127.21.55.65:45615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f5b4eeb6db7f40938f3454d05bba2035: Network error: Client connection negotiation failed: client connection to 127.21.55.65:45615: connect: Connection refused (error 111)
I20260810 14:41:22.169189 13875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:22.150853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43852 (local address 127.21.55.66:40337)
0810 14:41:22.151098 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:22.151105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:22.151295 (+ 190us) server_negotiation.cc:407] Connection header received
0810 14:41:22.151478 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:22.151484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:22.151581 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:22.151695 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:22.153703 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:22.154958 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:22.157125 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:22.157666 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:22.157812 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:22.161764 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:22.161784 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:22.161824 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:22.161893 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:22.165313 (+ 3420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:22.166316 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:22.166323 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:22.166325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:22.166489 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:22.167153 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:22.167159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:22.167162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:22.167911 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:22.168249 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:22.168628 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 14:41:22.168829 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 14:41:22.170046 13627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:22.205962 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:22.205449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45615 (local address 127.0.0.1:47458)
0810 14:41:22.205683 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:22.205793 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:45615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260810 14:41:22.207037 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:22.206728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45615 (local address 127.0.0.1:47466)
0810 14:41:22.206851 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:22.206931 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:45615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 14:41:22.267942 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:22.267404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45615 (local address 127.0.0.1:47470)
0810 14:41:22.267665 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:22.267761 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:45615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260810 14:41:22.269055 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:22.268697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45615 (local address 127.0.0.1:47486)
0810 14:41:22.268871 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:22.268947 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:45615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 14:41:22.299134 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 13434
I20260810 14:41:22.324849 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 13639
I20260810 14:41:22.343842 13878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:22.325726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47852 (local address 127.21.55.67:34635)
0810 14:41:22.325920 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:22.325926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:22.326001 (+ 75us) server_negotiation.cc:407] Connection header received
0810 14:41:22.326253 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:22.326258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:22.326349 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:22.326462 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:22.327841 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:22.328616 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:22.330001 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:22.330415 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:22.330595 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:22.337890 (+ 7295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:22.337906 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:22.337921 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:22.337956 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:22.340874 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:22.341647 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:22.341661 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:22.341663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:22.341806 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:22.342191 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:22.342196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:22.342201 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:22.342630 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:22.342933 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:22.343264 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 14:41:22.343500 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:41:22.344594 13839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:22.349879 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:22.349306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45615 (local address 127.0.0.1:47490)
0810 14:41:22.349556 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:22.349666 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:45615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 14:41:22.350929 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:22.350583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45615 (local address 127.0.0.1:47506)
0810 14:41:22.350751 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:22.350817 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:45615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 14:41:22.441890 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 13639
I20260810 14:41:22.469314 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 13085
I20260810 14:41:22.470788 13181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:22.587071 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 13085
I20260810 14:41:22.587208 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:22.586456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:45615 (local address 127.0.0.1:47510)
0810 14:41:22.586784 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:22.586955 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:45615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 14:41:22.615176 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38449
--webserver_interface=127.21.55.126
--webserver_port=45413
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:38449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:22.969154 13931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:22.969556 13931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:22.969740 13931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:22.977875 13931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:41:22.978235 13931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:22.978433 13931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:22.978582 13931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:41:22.978663 13931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:41:22.989480 13931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:38449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=45413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:22.991220 13931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:22.992658 13931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:23.004838 13941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:23.004801 13942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:23.005329 13944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:23.006454 13931 server_base.cc:1034] running on GCE node
I20260810 14:41:23.007651 13931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:23.009274 13931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:23.010591 13931 hybrid_clock.cc:648] HybridClock initialized: now 1786372883010467 us; error 74 us; skew 500 ppm
I20260810 14:41:23.010969 13931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:23 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372883, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:23.014827 13931 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:41:23.016422 13931 webserver.cc:460] Webserver started at http://127.21.55.126:45413/ using document root <none> and password file <none>
I20260810 14:41:23.017166 13931 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:23.017320 13931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:23.023085 13931 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260810 14:41:23.025959 13954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:23.027266 13931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:41:23.027441 13931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "d0eabdf2f3bb49d3991048f5981a7882"
format_stamp: "Formatted at 2026-08-10 14:41:16 on dist-test-slave-h6n0"
server_key: "afa95c2cbd4349e855aedc7a7ca75d9c"
server_key_iv: "c67b546f0ba3aeb73c8b81bd4036e037"
server_key_version: "encryptionkey@0"
I20260810 14:41:23.028416 13931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:23.049201 13931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:23.053642 13931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:23.053952 13931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:23.075271 13931 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:38449
I20260810 14:41:23.075299 14033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:38449 every 8 connection(s)
I20260810 14:41:23.076754 13931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:41:23.083643 14034 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:41:23.085644 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 13931
I20260810 14:41:23.086659 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:45615
--local_ip_for_outbound_sockets=127.21.55.65
--tserver_master_addrs=127.21.55.126:38449
--webserver_port=36389
--webserver_interface=127.21.55.65
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:41:23.087237 14034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Bootstrap starting.
I20260810 14:41:23.109287 14034 log.cc:824] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:23.126096 14034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:41:23.126724 14034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882: Bootstrap complete.
I20260810 14:41:23.137455 14034 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:23.138475 14034 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0eabdf2f3bb49d3991048f5981a7882, State: Initialized, Role: FOLLOWER
I20260810 14:41:23.139207 14034 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:23.139475 14034 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:41:23.139611 14034 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:41:23.139815 14034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 2 FOLLOWER]: Advancing to term 3
I20260810 14:41:23.143831 14034 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:23.144399 14034 leader_election.cc:302] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0eabdf2f3bb49d3991048f5981a7882; no voters:
I20260810 14:41:23.145154 14034 leader_election.cc:288] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 14:41:23.145298 14039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 14:41:23.146817 14039 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [term 3 LEADER]: Becoming Leader. State: Replica: d0eabdf2f3bb49d3991048f5981a7882, State: Running, Role: LEADER
I20260810 14:41:23.147935 14039 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } }
I20260810 14:41:23.148419 14034 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:41:23.151539 14043 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0eabdf2f3bb49d3991048f5981a7882. Latest consensus state: current_term: 3 leader_uuid: "d0eabdf2f3bb49d3991048f5981a7882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } } }
I20260810 14:41:23.153772 14043 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:23.151870 14040 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d0eabdf2f3bb49d3991048f5981a7882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0eabdf2f3bb49d3991048f5981a7882" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38449 } } }
I20260810 14:41:23.156298 14040 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0eabdf2f3bb49d3991048f5981a7882 [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:23.158388 14060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:41:23.174597 14060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:41:23.176950 14060 catalog_manager.cc:1259] Loaded cluster ID: 5106100e13c44ac88cfaec46a1f922b9
I20260810 14:41:23.177109 14060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 14:41:23.180310 14060 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f574c46e9ad google::LogMessage::Fail() at ??:0
@ 0x7f574c47188a google::LogMessage::SendToLog() at ??:0
I20260810 14:41:23.187600 14067 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f574c46e4de google::LogMessage::Flush() at ??:0
@ 0x7f574c46fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5761ea1512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f5761e9e976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5761f26610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f5761f26439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f574f0ba345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f574f0bd91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f574f0bd799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f574f098eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f57523726db start_thread at ??:0
@ 0x7f574b30861f clone at ??:0
W20260810 14:41:23.506165 14036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:23.506551 14036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:23.506736 14036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:23.520150 14036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:23.520396 14036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:23.520699 14036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:41:23.533164 14036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:45615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=36389
--tserver_master_addrs=127.21.55.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:23.534613 14036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:23.535964 14036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:23.551693 14097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:23.551728 14103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:23.551756 14098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:23.554030 14036 server_base.cc:1034] running on GCE node
I20260810 14:41:23.554839 14036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:23.556104 14036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:23.557400 14036 hybrid_clock.cc:648] HybridClock initialized: now 1786372883557367 us; error 99 us; skew 500 ppm
I20260810 14:41:23.557760 14036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:23 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372883, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:23.561285 14036 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:41:23.563050 14036 webserver.cc:460] Webserver started at http://127.21.55.65:36389/ using document root <none> and password file <none>
I20260810 14:41:23.564175 14036 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:23.564301 14036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:23.570024 14036 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.005s
I20260810 14:41:23.573527 14117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:23.576133 14036 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 14:41:23.576354 14036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "fe3aaa29a27343b6b9a6ab82892f22fe"
format_stamp: "Formatted at 2026-08-10 14:41:17 on dist-test-slave-h6n0"
server_key: "b16eb5c5f25cf51487f635ddea6ffafc"
server_key_iv: "6cd8fe03b09749364dc53e9231169e1d"
server_key_version: "encryptionkey@0"
I20260810 14:41:23.577481 14036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:23.599166 14036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:23.604357 14036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:23.604687 14036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:23.605784 14036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:23.607841 14125 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260810 14:41:23.613278 14125 ts_tablet_manager.cc:1852] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260810 14:41:23.614275 14125 ts_tablet_manager.cc:1905] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:23.621368 14125 ts_tablet_manager.cc:1918] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:41:23.621547 14125 log.cc:1196] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/f5b4eeb6db7f40938f3454d05bba2035
I20260810 14:41:23.622515 14125 ts_tablet_manager.cc:1939] T f5b4eeb6db7f40938f3454d05bba2035 P fe3aaa29a27343b6b9a6ab82892f22fe: Deleting consensus metadata
I20260810 14:41:23.623242 14036 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260810 14:41:23.623371 14036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260810 14:41:23.623544 14036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:23.623667 14036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:23.667373 14036 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:45615
I20260810 14:41:23.667395 14298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:45615 every 8 connection(s)
I20260810 14:41:23.669437 14036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:41:23.675079 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:23.673053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:42599)
0810 14:41:23.673860 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:23.674115 (+ 255us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":355,"threads_started":1}
W20260810 14:41:23.675846 14299 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:38449 (0 consecutive failures): Network error: Failed to ping master at 127.21.55.126:38449: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
I20260810 14:41:23.676885 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:23.676498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:34127)
0810 14:41:23.676686 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:23.676770 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 14:41:23.678090 14308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:23.677707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:55451)
0810 14:41:23.677889 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:23.677966 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260810 14:41:23.678452 14299 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:41:23.678887 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 14036
I20260810 14:41:23.679523 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:40337
--local_ip_for_outbound_sockets=127.21.55.66
--tserver_master_addrs=127.21.55.126:38449
--webserver_port=33569
--webserver_interface=127.21.55.66
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:24.056689 14309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:24.057067 14309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:24.057264 14309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:24.064746 14309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:24.064931 14309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:24.065136 14309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:41:24.075302 14309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:40337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=33569
--tserver_master_addrs=127.21.55.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:24.076730 14309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:24.078181 14309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:24.095150 14318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:24.095162 14321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:24.095139 14319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:24.095876 14309 server_base.cc:1034] running on GCE node
I20260810 14:41:24.096638 14309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:24.097625 14309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:24.098912 14309 hybrid_clock.cc:648] HybridClock initialized: now 1786372884098854 us; error 72 us; skew 500 ppm
I20260810 14:41:24.099311 14309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:24 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372884, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:24.102967 14309 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:41:24.104568 14309 webserver.cc:460] Webserver started at http://127.21.55.66:33569/ using document root <none> and password file <none>
I20260810 14:41:24.105247 14309 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:24.105351 14309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:24.111321 14309 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:41:24.114584 14332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:24.116052 14309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:41:24.116201 14309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "c833c6dff13c470a8a4e16a88062202f"
format_stamp: "Formatted at 2026-08-10 14:41:17 on dist-test-slave-h6n0"
server_key: "671000839c2305d07ec6e3dd40927c9e"
server_key_iv: "d1fcdfd0759c7f95519f18a60efe2cf2"
server_key_version: "encryptionkey@0"
I20260810 14:41:24.117218 14309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:24.133421 14309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:24.138257 14309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:24.138628 14309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:24.139643 14309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:24.141331 14309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:24.141441 14309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:24.141574 14309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:24.141661 14309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:24.185729 14309 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:40337
I20260810 14:41:24.185838 14496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:40337 every 8 connection(s)
I20260810 14:41:24.188712 14309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:41:24.194052 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 14309
I20260810 14:41:24.194986 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:34635
--local_ip_for_outbound_sockets=127.21.55.67
--tserver_master_addrs=127.21.55.126:38449
--webserver_port=38071
--webserver_interface=127.21.55.67
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:41:24.195107 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:24.192481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:34625)
0810 14:41:24.193347 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:24.193747 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":362,"threads_started":1}
W20260810 14:41:24.195967 14497 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:38449 (0 consecutive failures): Network error: Failed to ping master at 127.21.55.126:38449: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
I20260810 14:41:24.197065 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:24.196567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:34845)
0810 14:41:24.196785 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:24.196892 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 14:41:24.198179 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:24.197709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:48147)
0810 14:41:24.197902 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:24.198012 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 14:41:24.198501 14497 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:41:24.578265 14503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:24.578666 14503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:24.578871 14503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:24.587361 14503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:24.587554 14503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:24.587793 14503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:41:24.599938 14503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:34635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=38071
--tserver_master_addrs=127.21.55.126:38449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:24.601446 14503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:24.602912 14503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:24.619728 14508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:24.619822 14511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:24.619855 14513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:24.621486 14503 server_base.cc:1034] running on GCE node
I20260810 14:41:24.622291 14503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:24.623435 14503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:24.624681 14503 hybrid_clock.cc:648] HybridClock initialized: now 1786372884624615 us; error 47 us; skew 500 ppm
I20260810 14:41:24.625088 14503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:24 dist-test-slave-h6n0 krb5kdc[12230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372884, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:24.629477 14503 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:41:24.634366 14503 webserver.cc:460] Webserver started at http://127.21.55.67:38071/ using document root <none> and password file <none>
I20260810 14:41:24.637066 14503 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:24.637638 14503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:24.646507 14503 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 14:41:24.649791 14526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:24.651396 14503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:41:24.651616 14503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "9a09ae3be792404dbfe7df68e7db7305"
format_stamp: "Formatted at 2026-08-10 14:41:18 on dist-test-slave-h6n0"
server_key: "5ac5c607c761c46eef096a6d15e960db"
server_key_iv: "f61a722f0f07c31ad2ea447519961038"
server_key_version: "encryptionkey@0"
I20260810 14:41:24.652669 14503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:24.671221 14503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:24.676466 14503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:24.676795 14503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:24.677901 14503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:24.679654 14503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:24.679826 14503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 14:41:24.679967 14503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:24.680063 14503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:24.680315 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:24.679320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:42337)
0810 14:41:24.679830 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:24.679968 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":231,"threads_started":1}
I20260810 14:41:24.725746 14503 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:34635
I20260810 14:41:24.725880 14700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:34635 every 8 connection(s)
I20260810 14:41:24.727387 14503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:41:24.728587 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 14503
I20260810 14:41:24.734210 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:24.731778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:53815)
0810 14:41:24.732952 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:24.733365 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":343,"threads_started":1}
W20260810 14:41:24.735266 14702 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:38449 (0 consecutive failures): Network error: Failed to ping master at 127.21.55.126:38449: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
I20260810 14:41:24.736325 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:24.735924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:48287)
0810 14:41:24.736132 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:24.736212 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 14:41:24.737884 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:24.737497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:51407)
0810 14:41:24.737672 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:24.737758 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260810 14:41:24.738304 14702 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:41:25.200595 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:25.199393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:59881)
0810 14:41:25.200171 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:25.200314 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":359,"threads_started":1}
I20260810 14:41:25.682356 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:25.681485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:42423)
0810 14:41:25.681937 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:25.682030 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":244,"threads_started":1}
I20260810 14:41:25.740682 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:25.739289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:52015)
0810 14:41:25.740185 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:25.740368 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":561,"threads_started":1}
I20260810 14:41:26.202405 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:26.201640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:50089)
0810 14:41:26.202130 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:26.202229 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":250,"threads_started":1}
I20260810 14:41:26.684907 14714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:26.683798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:57845)
0810 14:41:26.684605 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:26.684726 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":508,"threads_started":1}
I20260810 14:41:26.742934 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:26.741968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:58355)
0810 14:41:26.742506 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:26.742715 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":277,"threads_started":1}
I20260810 14:41:27.205088 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:27.203544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:45249)
0810 14:41:27.204320 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:27.204731 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":384,"threads_started":1}
I20260810 14:41:27.687569 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:27.686494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:42833)
0810 14:41:27.687204 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:27.687307 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":391,"threads_started":1}
I20260810 14:41:27.744948 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:27.744096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:52873)
0810 14:41:27.744661 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:27.744754 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":284,"threads_started":1}
I20260810 14:41:28.207350 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:28.206375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:42467)
0810 14:41:28.207013 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:28.207120 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":349,"threads_started":1}
I20260810 14:41:28.690075 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:28.688914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:40083)
0810 14:41:28.689708 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:28.689844 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":493,"threads_started":1}
I20260810 14:41:28.747188 14723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:28.746188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:34597)
0810 14:41:28.746917 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:28.747004 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":359,"threads_started":1}
I20260810 14:41:29.209991 14725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:29.208800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:39001)
0810 14:41:29.209440 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:29.209534 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":342,"threads_started":1}
I20260810 14:41:29.692376 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:29.691532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:55519)
0810 14:41:29.692089 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:29.692188 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":297,"threads_started":1}
I20260810 14:41:29.750087 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:29.749053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:50945)
0810 14:41:29.749770 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:29.749888 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":391,"threads_started":1}
I20260810 14:41:30.212445 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:30.211593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:49001)
0810 14:41:30.212163 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:30.212267 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":289,"threads_started":1}
I20260810 14:41:30.695022 14731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:30.693587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:48907)
0810 14:41:30.694500 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:30.694729 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":338,"threads_started":1}
I20260810 14:41:30.753013 14732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:30.751655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:40647)
0810 14:41:30.752702 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:30.752820 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":911,"thread_start_us":701,"threads_started":1}
I20260810 14:41:31.215210 14733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:31.214144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:36385)
0810 14:41:31.214817 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:31.214954 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":359,"threads_started":1}
I20260810 14:41:31.697077 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:31.696224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:37917)
0810 14:41:31.696757 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:31.696878 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":297,"threads_started":1}
I20260810 14:41:31.755201 14735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:31.754252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:48445)
0810 14:41:31.754868 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:31.754968 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":325,"threads_started":1}
I20260810 14:41:32.217780 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:32.216921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:52997)
0810 14:41:32.217502 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:32.217598 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":296,"threads_started":1}
I20260810 14:41:32.699914 14737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:32.698641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:56435)
0810 14:41:32.699490 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:32.699651 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":341,"threads_started":1}
I20260810 14:41:32.757838 14738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:32.756788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:38099)
0810 14:41:32.757517 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:32.757638 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":382,"threads_started":1}
I20260810 14:41:33.220150 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:33.219108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:37591)
0810 14:41:33.219813 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:33.219922 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":402,"threads_started":1}
I20260810 14:41:33.703004 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:33.701816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:59701)
0810 14:41:33.702638 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:33.702791 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":523,"threads_started":1}
I20260810 14:41:33.760375 14743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:33.759235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:36823)
0810 14:41:33.759810 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:33.759929 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":302,"threads_started":1}
I20260810 14:41:34.222992 14746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:34.221959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:41357)
0810 14:41:34.222518 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:34.222785 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":331,"threads_started":1}
I20260810 14:41:34.705297 14748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:34.704261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:58045)
0810 14:41:34.704919 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:34.705033 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":394,"threads_started":1}
I20260810 14:41:34.762687 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:34.761672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:59973)
0810 14:41:34.762355 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:34.762476 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":394,"threads_started":1}
I20260810 14:41:35.225582 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:35.224459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:58571)
0810 14:41:35.225196 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:35.225335 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":326,"threads_started":1}
I20260810 14:41:35.707377 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:35.706560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:45225)
0810 14:41:35.707099 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:35.707198 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":300,"threads_started":1}
I20260810 14:41:35.764794 14756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:35.763929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:48755)
0810 14:41:35.764481 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:35.764599 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":332,"threads_started":1}
I20260810 14:41:36.228295 14757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:36.227325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:43405)
0810 14:41:36.227952 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:36.228103 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":335,"threads_started":1}
I20260810 14:41:36.710459 14758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:36.708870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:48751)
0810 14:41:36.709815 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:36.710034 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":504,"threads_started":1}
I20260810 14:41:36.767411 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:36.766356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:45579)
0810 14:41:36.767068 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:36.767178 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":381,"threads_started":1}
I20260810 14:41:37.230736 14760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:37.229679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:39339)
0810 14:41:37.230337 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:37.230437 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":411,"threads_started":1}
I20260810 14:41:37.712723 14766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:37.711579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:60451)
0810 14:41:37.712323 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:37.712457 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":365,"threads_started":1}
I20260810 14:41:37.769636 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:37.768756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:51733)
0810 14:41:37.769315 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:37.769428 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":296,"threads_started":1}
I20260810 14:41:38.233741 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:38.232455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:33351)
0810 14:41:38.233333 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:38.233477 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":316,"threads_started":1}
I20260810 14:41:38.715824 14770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:38.714683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:45141)
0810 14:41:38.715460 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:38.715576 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":446,"threads_started":1}
I20260810 14:41:38.772282 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:38.770872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:47371)
0810 14:41:38.771893 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:38.772032 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":440,"threads_started":1}
I20260810 14:41:39.236932 14774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:39.235675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.66:45367)
0810 14:41:39.236433 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:39.236673 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":423,"threads_started":1}
I20260810 14:41:39.718514 14775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:39.717418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.65:37793)
0810 14:41:39.718178 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:39.718305 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":404,"threads_started":1}
I20260810 14:41:39.734334 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 14036
I20260810 14:41:39.766986 14778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:39.736533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.21.55.65:45615)
0810 14:41:39.737949 (+ 1416us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:39.737971 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:39.738026 (+ 55us) server_negotiation.cc:407] Connection header received
0810 14:41:39.740382 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:39.740400 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:39.741079 (+ 679us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:39.741591 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:39.743806 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:39.745394 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:39.747327 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:39.748268 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:39.748533 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:39.757136 (+ 8603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:39.757150 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:39.757175 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:39.757250 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:39.761801 (+ 4551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:39.763075 (+ 1274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:39.763085 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:39.763097 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:39.763413 (+ 316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:39.763947 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:39.763951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:39.763954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:39.764775 (+ 821us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:39.765249 (+ 474us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:39.765826 (+ 577us) server_negotiation.cc:299] Negotiation successful
0810 14:41:39.766486 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1128,"thread_start_us":855,"threads_started":1}
I20260810 14:41:39.768846 14289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:39.775748 14782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:39.773794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38449 (local address 127.21.55.67:33375)
0810 14:41:39.775337 (+ 1543us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:39.775493 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:38449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1210,"thread_start_us":999,"threads_started":1}
I20260810 14:41:39.873128 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 14036
I20260810 14:41:39.908259 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 14309
I20260810 14:41:39.935863 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:39.909843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.21.55.66:40337)
0810 14:41:39.910673 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:39.910692 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:39.910745 (+ 53us) server_negotiation.cc:407] Connection header received
0810 14:41:39.912698 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:39.912710 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:39.913314 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:39.913717 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:39.915758 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:39.917603 (+ 1845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:39.919842 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:39.920849 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:39.921272 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:39.925639 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:39.925652 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:39.925677 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:39.925752 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:39.929724 (+ 3972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:39.931507 (+ 1783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:39.931519 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:39.931528 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:39.931811 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:39.932614 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:39.932618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:39.932621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:39.933471 (+ 850us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:39.933801 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:39.934666 (+ 865us) server_negotiation.cc:299] Negotiation successful
0810 14:41:39.935480 (+ 814us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":388,"threads_started":1}
I20260810 14:41:39.937836 14490 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:40.042501 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 14309
I20260810 14:41:40.074707 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 14503
I20260810 14:41:40.097808 14787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:40.076200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40778 (local address 127.21.55.67:34635)
0810 14:41:40.077036 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:40.077051 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:40.077100 (+ 49us) server_negotiation.cc:407] Connection header received
0810 14:41:40.078935 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:40.078944 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:40.079403 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:40.079783 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:40.081381 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:40.082733 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:40.084084 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:40.084785 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:40.085153 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:40.088421 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:40.088430 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:40.088447 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:40.088497 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:40.092215 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:40.093054 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:40.093062 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:40.093153 (+ 91us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:40.093581 (+ 428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:40.094305 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:40.094316 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:40.094319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:40.095258 (+ 939us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:40.095731 (+ 473us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:40.096509 (+ 778us) server_negotiation.cc:299] Negotiation successful
0810 14:41:40.097362 (+ 853us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":420,"threads_started":1}
I20260810 14:41:40.099874 14696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:40.208048 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 14503
2026-08-10T14:41:40Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25833 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:41:40 dist-test-slave-h6n0 krb5kdc[14791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:41:40 dist-test-slave-h6n0 krb5kdc[14791](info): set up 2 sockets
Aug 10 14:41:40 dist-test-slave-h6n0 krb5kdc[14791](info): commencing operation
krb5kdc: starting...
W20260810 14:41:41.793169 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.499s user 0.010s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:41:42 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:41:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:41:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:41:42.139355 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:43661
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:43661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:42.539297 14813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:42.539657 14813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:42.539835 14813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:42.547358 14813 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 14:41:42.547492 14813 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 14:41:42.547583 14813 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 14:41:42.547667 14813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:41:42.547750 14813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:42.547871 14813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:42.547943 14813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:41:42.548020 14813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:41:42.559887 14813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:43661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:43661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:42.561317 14813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:42.562696 14813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:42.577368 14823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:42.577304 14824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:42.577308 14828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:42.578337 14813 server_base.cc:1034] running on GCE node
I20260810 14:41:42.579165 14813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:42.580564 14813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:42.581845 14813 hybrid_clock.cc:648] HybridClock initialized: now 1786372902581778 us; error 39 us; skew 500 ppm
I20260810 14:41:42.582221 14813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:42 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372902, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:42.586706 14813 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:41:42.588542 14813 webserver.cc:460] Webserver started at http://127.21.55.126:34265/ using document root <none> and password file <none>
I20260810 14:41:42.589272 14813 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:42.589413 14813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:42.589752 14813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:42.592783 14813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "d0bf2c0428fb4615800a8e628b37861d"
format_stamp: "Formatted at 2026-08-10 14:41:42 on dist-test-slave-h6n0"
server_key: "bf162931067409522fbb674d551c9efd"
server_key_iv: "936bc9fe80a91f7bb36fe495287257b6"
server_key_version: "encryptionkey@0"
I20260810 14:41:42.593608 14813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "d0bf2c0428fb4615800a8e628b37861d"
format_stamp: "Formatted at 2026-08-10 14:41:42 on dist-test-slave-h6n0"
server_key: "bf162931067409522fbb674d551c9efd"
server_key_iv: "936bc9fe80a91f7bb36fe495287257b6"
server_key_version: "encryptionkey@0"
I20260810 14:41:42.599776 14813 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 14:41:42.604364 14837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:42.606060 14813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260810 14:41:42.606271 14813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "d0bf2c0428fb4615800a8e628b37861d"
format_stamp: "Formatted at 2026-08-10 14:41:42 on dist-test-slave-h6n0"
server_key: "bf162931067409522fbb674d551c9efd"
server_key_iv: "936bc9fe80a91f7bb36fe495287257b6"
server_key_version: "encryptionkey@0"
I20260810 14:41:42.606530 14813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:42.633514 14813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:42.640021 14813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:42.640424 14813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:42.666067 14813 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:43661
I20260810 14:41:42.666419 14910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:43661 every 8 connection(s)
I20260810 14:41:42.667557 14813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:41:42.671430 14914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:42.677546 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 14813
I20260810 14:41:42.678150 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:41:42.679612 21725 external_mini_cluster.cc:1428] Setting key 953c031b2c5e237805914d677f36b4d7
I20260810 14:41:42.684886 14914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Bootstrap starting.
I20260810 14:41:42.688522 14914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:42.689926 14914 log.cc:824] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:42.692865 14914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: No bootstrap required, opened a new log
Aug 10 14:41:42 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:42.702461 14914 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:41:42.703173 14914 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:42.703334 14914 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0bf2c0428fb4615800a8e628b37861d, State: Initialized, Role: FOLLOWER
I20260810 14:41:42.704258 14914 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:41:42.704669 14914 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:41:42.704865 14914 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:41:42.705096 14914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:42.707567 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:42.683332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.21.55.126:43661)
0810 14:41:42.684523 (+ 1191us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:42.684537 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:42.684583 (+ 46us) server_negotiation.cc:407] Connection header received
0810 14:41:42.686802 (+ 2219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:42.686826 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:42.687217 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:42.687542 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:42.689684 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:42.691196 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:42.693422 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:42.694415 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:42.694740 (+ 325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:42.699455 (+ 4715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:42.699464 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:42.699483 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:42.699525 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:42.702915 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:42.703760 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:42.703768 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:42.703774 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:42.703977 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:42.704388 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:42.704394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:42.704399 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:42.705105 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:42.705467 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:42.705922 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 14:41:42.706411 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":914,"thread_start_us":508,"threads_started":1}
I20260810 14:41:42.708209 14914 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:41:42.708832 14914 leader_election.cc:302] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0bf2c0428fb4615800a8e628b37861d; no voters:
I20260810 14:41:42.709501 14914 leader_election.cc:288] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:41:42.709650 14923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:41:42.711230 14923 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 1 LEADER]: Becoming Leader. State: Replica: d0bf2c0428fb4615800a8e628b37861d, State: Running, Role: LEADER
I20260810 14:41:42.711774 14923 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:41:42.712155 14914 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:41:42.715149 14927 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0bf2c0428fb4615800a8e628b37861d. Latest consensus state: current_term: 1 leader_uuid: "d0bf2c0428fb4615800a8e628b37861d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } } }
I20260810 14:41:42.715952 14927 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:42.715126 14925 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0bf2c0428fb4615800a8e628b37861d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } } }
I20260810 14:41:42.717222 14925 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: This master's current role is: LEADER
I20260810 14:41:42.718551 14933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:41:42.725805 14933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:41:42.735823 14933 catalog_manager.cc:1347] Generated new cluster ID: 3fcdc3251e3f46a1b7ee2c0a60c5b4e5
I20260810 14:41:42.736069 14933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:41:42.752151 14933 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:41:42.754609 14933 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:41:42.760643 14933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Generated new TSK 0
I20260810 14:41:42.761480 14933 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:41:42.861467 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43661
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:43.256996 14954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:43.257318 14954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:43.257450 14954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:43.266999 14954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:43.267146 14954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:43.267282 14954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:41:43.279259 14954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:43.280563 14954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:43.281929 14954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:43.297088 14961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:43.297282 14960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:43.297217 14963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:43.299480 14954 server_base.cc:1034] running on GCE node
I20260810 14:41:43.300532 14954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:43.301700 14954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:43.302965 14954 hybrid_clock.cc:648] HybridClock initialized: now 1786372903302880 us; error 63 us; skew 500 ppm
I20260810 14:41:43.303344 14954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:43 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372903, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:43.307374 14954 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:41:43.309114 14954 webserver.cc:460] Webserver started at http://127.21.55.65:37441/ using document root <none> and password file <none>
I20260810 14:41:43.309855 14954 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:43.310017 14954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:43.310392 14954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:43.313774 14954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "580e0549c0b241dd8865e230d10963a9"
format_stamp: "Formatted at 2026-08-10 14:41:43 on dist-test-slave-h6n0"
server_key: "5c107c5a5ddd9780191ffc9d6a33adef"
server_key_iv: "5771bec2725c739cc1e4e193b49968f2"
server_key_version: "encryptionkey@0"
I20260810 14:41:43.314647 14954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "580e0549c0b241dd8865e230d10963a9"
format_stamp: "Formatted at 2026-08-10 14:41:43 on dist-test-slave-h6n0"
server_key: "5c107c5a5ddd9780191ffc9d6a33adef"
server_key_iv: "5771bec2725c739cc1e4e193b49968f2"
server_key_version: "encryptionkey@0"
I20260810 14:41:43.320604 14954 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260810 14:41:43.325258 14979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:43.327414 14954 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 14:41:43.327639 14954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "580e0549c0b241dd8865e230d10963a9"
format_stamp: "Formatted at 2026-08-10 14:41:43 on dist-test-slave-h6n0"
server_key: "5c107c5a5ddd9780191ffc9d6a33adef"
server_key_iv: "5771bec2725c739cc1e4e193b49968f2"
server_key_version: "encryptionkey@0"
I20260810 14:41:43.327900 14954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:43.354756 14954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:43.359790 14954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:43.360080 14954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:43.361195 14954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:43.362673 14954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:43.362758 14954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:43.362854 14954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:43.362914 14954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:43.408380 14954 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:46259
I20260810 14:41:43.408452 15155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:46259 every 8 connection(s)
I20260810 14:41:43.409937 14954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:41:43.420523 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 14954
I20260810 14:41:43.420779 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:41:43.421703 21725 external_mini_cluster.cc:1428] Setting key 763a567077f7bdaa3335d6b7401987c5
Aug 10 14:41:43 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372903, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:43.439579 15159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:43.414060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:54707 (local address 127.21.55.126:43661)
0810 14:41:43.414664 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:43.414668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:43.416390 (+ 1722us) server_negotiation.cc:407] Connection header received
0810 14:41:43.418762 (+ 2372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:43.418769 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:43.418892 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:43.419029 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:43.421093 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:43.422230 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:43.424616 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:43.425288 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:43.425445 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:43.431197 (+ 5752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:43.431207 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:43.431215 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:43.431264 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:43.435391 (+ 4127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:43.436543 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:43.436552 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:43.436558 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:43.436720 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:43.437368 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:43.437374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:43.437378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:43.437794 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:43.437960 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:43.438807 (+ 847us) server_negotiation.cc:299] Negotiation successful
0810 14:41:43.439071 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":301,"threads_started":1}
I20260810 14:41:43.440332 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:43.414843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:54707)
0810 14:41:43.416184 (+ 1341us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:43.416234 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:43.418409 (+ 2175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:43.419304 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:43.419313 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:43.420071 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:43.420865 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:43.420895 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:43.422478 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:43.422486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:43.424445 (+ 1959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:43.424462 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:43.425572 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:43.425578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:43.425819 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:43.426977 (+ 1158us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:43.427016 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:43.430887 (+ 3871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:43.435641 (+ 4754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:43.435652 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:43.435670 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:43.436349 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:43.436910 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:43.436915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:43.436918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:43.437214 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:43.437975 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:43.437982 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:43.438344 (+ 362us) client_negotiation.cc:769] Sending connection context
0810 14:41:43.438666 (+ 322us) client_negotiation.cc:240] Negotiation successful
0810 14:41:43.439124 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"thread_start_us":600,"threads_started":1}
I20260810 14:41:43.442489 15156 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43661
I20260810 14:41:43.443091 15156 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:43.443980 15156 heartbeater.cc:511] Master 127.21.55.126:43661 requested a full tablet report, sending...
I20260810 14:41:43.447485 14863 ts_manager.cc:194] Registered new tserver with Master: 580e0549c0b241dd8865e230d10963a9 (127.21.55.65:46259)
I20260810 14:41:43.449796 14863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:54707
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:41:43.498055 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43661
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:41:43.890753 15166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:43.891108 15166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:43.891287 15166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:43.898622 15166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:43.898780 15166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:43.898967 15166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:41:43.908998 15166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:43.910561 15166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:43.912082 15166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:43.927420 15178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:43.927563 15172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:43.927469 15174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:43.928996 15166 server_base.cc:1034] running on GCE node
I20260810 14:41:43.930059 15166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:43.931685 15166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:43.933024 15166 hybrid_clock.cc:648] HybridClock initialized: now 1786372903932878 us; error 95 us; skew 500 ppm
I20260810 14:41:43.933671 15166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:43 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372903, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:43.937618 15166 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:41:43.939566 15166 webserver.cc:460] Webserver started at http://127.21.55.66:37665/ using document root <none> and password file <none>
I20260810 14:41:43.940346 15166 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:43.940516 15166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:43.940938 15166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:43.944344 15166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "2a60da52205e49b09d90da0b303ce671"
format_stamp: "Formatted at 2026-08-10 14:41:43 on dist-test-slave-h6n0"
server_key: "9ba922d800e835b89376f3b9fc8feb4e"
server_key_iv: "2c2fe987edbf123f089aa0d33f918113"
server_key_version: "encryptionkey@0"
I20260810 14:41:43.945120 15166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "2a60da52205e49b09d90da0b303ce671"
format_stamp: "Formatted at 2026-08-10 14:41:43 on dist-test-slave-h6n0"
server_key: "9ba922d800e835b89376f3b9fc8feb4e"
server_key_iv: "2c2fe987edbf123f089aa0d33f918113"
server_key_version: "encryptionkey@0"
I20260810 14:41:43.950984 15166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260810 14:41:43.955273 15186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:43.957105 15166 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 14:41:43.957316 15166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "2a60da52205e49b09d90da0b303ce671"
format_stamp: "Formatted at 2026-08-10 14:41:43 on dist-test-slave-h6n0"
server_key: "9ba922d800e835b89376f3b9fc8feb4e"
server_key_iv: "2c2fe987edbf123f089aa0d33f918113"
server_key_version: "encryptionkey@0"
I20260810 14:41:43.957538 15166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:43.993391 15166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:43.999651 15166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:44.000021 15166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:44.001102 15166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:44.003024 15166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:44.003299 15166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:44.003515 15166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:44.003611 15166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:44.047286 15166 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:33019
I20260810 14:41:44.047344 15351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:33019 every 8 connection(s)
I20260810 14:41:44.049671 15166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:41:44.052112 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 15166
I20260810 14:41:44.052376 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:41:44.053063 21725 external_mini_cluster.cc:1428] Setting key b18308f22ac21f92b95cd993d6a5c164
Aug 10 14:41:44 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372903, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:44.074273 15355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:44.053130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:44359 (local address 127.21.55.126:43661)
0810 14:41:44.053719 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:44.053726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:44.054943 (+ 1217us) server_negotiation.cc:407] Connection header received
0810 14:41:44.057018 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:44.057025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:44.057133 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:44.057265 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:44.058930 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.059734 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.061695 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.062308 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.062454 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:44.067472 (+ 5018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:44.067477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:44.067480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:44.067516 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:44.070390 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:44.071540 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:44.071544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:44.071546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:44.071672 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:44.072277 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:44.072281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:44.072283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:44.072592 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:44.072731 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:44.073793 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0810 14:41:44.073968 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":291,"threads_started":1}
I20260810 14:41:44.076248 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:44.053741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:44359)
0810 14:41:44.054759 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:44.054808 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:44.056736 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:44.057525 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:44.057534 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:44.058090 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:44.058750 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:44.058770 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.059927 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.059931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:44.061537 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:44.061558 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.062607 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.062617 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:44.063279 (+ 662us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:44.064158 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:44.064191 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:44.067185 (+ 2994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:44.070585 (+ 3400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:44.070598 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:44.070619 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:44.071371 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:44.071807 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:44.071815 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:44.071831 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:44.072119 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:44.072753 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:44.072765 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:44.073336 (+ 571us) client_negotiation.cc:769] Sending connection context
0810 14:41:44.073675 (+ 339us) client_negotiation.cc:240] Negotiation successful
0810 14:41:44.074321 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":436,"threads_started":1}
I20260810 14:41:44.078720 15352 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43661
I20260810 14:41:44.079180 15352 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:44.080159 15352 heartbeater.cc:511] Master 127.21.55.126:43661 requested a full tablet report, sending...
I20260810 14:41:44.082139 14863 ts_manager.cc:194] Registered new tserver with Master: 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019)
I20260810 14:41:44.083094 14863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:44359
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:41:44.123772 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43661
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:41:44.454288 15156 heartbeater.cc:503] Master 127.21.55.126:43661 was elected leader, sending a full tablet report...
W20260810 14:41:44.503844 15362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:41:44.504146 15362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:41:44.504316 15362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:41:44.511708 15362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:41:44.511816 15362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:41:44.511929 15362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:41:44.521750 15362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:41:44.523161 15362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:41:44.524469 15362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:41:44.538394 15373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:44.538904 15374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:41:44.539306 15376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:41:44.540975 15362 server_base.cc:1034] running on GCE node
I20260810 14:41:44.541646 15362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:41:44.542817 15362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:41:44.544077 15362 hybrid_clock.cc:648] HybridClock initialized: now 1786372904544040 us; error 45 us; skew 500 ppm
I20260810 14:41:44.544446 15362 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:41:44 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372904, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:41:44.547943 15362 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:41:44.549475 15362 webserver.cc:460] Webserver started at http://127.21.55.67:35107/ using document root <none> and password file <none>
I20260810 14:41:44.550169 15362 fs_manager.cc:359] Metadata directory not provided
I20260810 14:41:44.550278 15362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:41:44.550591 15362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:41:44.553648 15362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "9fbff42dc4b243f39747ee9030672130"
format_stamp: "Formatted at 2026-08-10 14:41:44 on dist-test-slave-h6n0"
server_key: "1aa03bacc9924dc835558d331b4db2d0"
server_key_iv: "fe7d746c6231f4ef3ada432848ddbecb"
server_key_version: "encryptionkey@0"
I20260810 14:41:44.554412 15362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "9fbff42dc4b243f39747ee9030672130"
format_stamp: "Formatted at 2026-08-10 14:41:44 on dist-test-slave-h6n0"
server_key: "1aa03bacc9924dc835558d331b4db2d0"
server_key_iv: "fe7d746c6231f4ef3ada432848ddbecb"
server_key_version: "encryptionkey@0"
I20260810 14:41:44.560261 15362 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260810 14:41:44.564127 15387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:44.565804 15362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:41:44.565999 15362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "9fbff42dc4b243f39747ee9030672130"
format_stamp: "Formatted at 2026-08-10 14:41:44 on dist-test-slave-h6n0"
server_key: "1aa03bacc9924dc835558d331b4db2d0"
server_key_iv: "fe7d746c6231f4ef3ada432848ddbecb"
server_key_version: "encryptionkey@0"
I20260810 14:41:44.566227 15362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:41:44.588335 15362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:41:44.593039 15362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:41:44.593398 15362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:41:44.594715 15362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:41:44.596235 15362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:41:44.596318 15362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:44.596406 15362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:41:44.596517 15362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:41:44.640818 15362 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:36769
I20260810 14:41:44.641070 15549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:36769 every 8 connection(s)
I20260810 14:41:44.642453 15362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:41:44.651036 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 15362
I20260810 14:41:44.651279 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:41:44.651798 21725 external_mini_cluster.cc:1428] Setting key 308a1186e3b867e21f7fa719316798fa
Aug 10 14:41:44 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372904, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:41:44.667615 15554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:44.645352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:48563 (local address 127.21.55.126:43661)
0810 14:41:44.645877 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:44.645904 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:44.647026 (+ 1122us) server_negotiation.cc:407] Connection header received
0810 14:41:44.649017 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:44.649023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:44.649127 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:44.649252 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:44.650928 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.651833 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.653671 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.654198 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.654335 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:44.659284 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:44.659289 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:44.659292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:44.659327 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:44.662984 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:44.663841 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:44.663846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:44.663848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:44.663980 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:44.664587 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:44.664595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:44.664597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:44.665149 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:44.665343 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:44.666608 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0810 14:41:44.667078 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":274,"threads_started":1}
I20260810 14:41:44.668720 15555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:44.645698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:48563)
0810 14:41:44.646857 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:44.646911 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:44.648738 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:44.649510 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:44.649518 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:44.650033 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:41:44.650757 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:44.650784 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.651996 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.652001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:44.653523 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:44.653542 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.654424 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.654433 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:44.654738 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:44.655618 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:41:44.655686 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:41:44.659015 (+ 3329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:41:44.663186 (+ 4171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:44.663194 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:44.663208 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:44.663706 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:44.664112 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:41:44.664116 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:41:44.664119 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:41:44.664325 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:41:44.665427 (+ 1102us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:41:44.665435 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:41:44.665954 (+ 519us) client_negotiation.cc:769] Sending connection context
0810 14:41:44.666396 (+ 442us) client_negotiation.cc:240] Negotiation successful
0810 14:41:44.667149 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"thread_start_us":527,"threads_started":1}
I20260810 14:41:44.670210 15550 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43661
I20260810 14:41:44.670598 15550 heartbeater.cc:464] Registering TS with master...
I20260810 14:41:44.671495 15550 heartbeater.cc:511] Master 127.21.55.126:43661 requested a full tablet report, sending...
I20260810 14:41:44.673038 14863 ts_manager.cc:194] Registered new tserver with Master: 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769)
I20260810 14:41:44.673918 14863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:48563
I20260810 14:41:44.683815 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:41:44.705862 15554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:44.689315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.21.55.126:43661)
0810 14:41:44.689659 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:44.689666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:44.690042 (+ 376us) server_negotiation.cc:407] Connection header received
0810 14:41:44.690109 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:44.690113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:44.690194 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:44.690285 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:44.692287 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.693128 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.695132 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.695559 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.695684 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:44.699159 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:44.699166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:44.699176 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:44.699227 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:44.702513 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:44.703467 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:44.703471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:44.703473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:44.703606 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:44.704227 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:44.704231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:44.704233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:44.704614 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:44.704827 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:44.705273 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 14:41:44.705493 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260810 14:41:44.711728 14863 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:41:44.719102 14863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:41:44.746570 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:44.735081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.0.0.1:50068)
0810 14:41:44.735960 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:44.736002 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:44.736245 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:44.737223 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:44.737230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:44.737269 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:44.737680 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:44.737694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.739563 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.739571 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:44.742392 (+ 2821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:44.742413 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.745254 (+ 2841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.745261 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:44.745501 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:44.745559 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 14:41:44.745697 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 14:41:44.745993 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":428,"threads_started":1}
I20260810 14:41:44.748006 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:44.735224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.21.55.65:46259)
0810 14:41:44.735979 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:44.735988 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:44.736128 (+ 140us) server_negotiation.cc:407] Connection header received
0810 14:41:44.736544 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:44.736551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:44.736734 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:44.736925 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:44.737904 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.739399 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.742649 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.744694 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.744837 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:44.745119 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:44.747266 (+ 2147us) server_negotiation.cc:299] Negotiation successful
0810 14:41:44.747529 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":413,"threads_started":1}
I20260810 14:41:44.757980 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:44.739407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:33019 (local address 127.0.0.1:37746)
0810 14:41:44.746906 (+ 7499us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:44.746930 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:44.747063 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:44.749286 (+ 2223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:44.749291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:44.749319 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:44.749601 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:44.749615 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.752782 (+ 3167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.752793 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:44.755051 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:44.755096 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.757117 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.757126 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:44.757333 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:44.757375 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 14:41:44.757450 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 14:41:44.757613 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7334,"mutex_wait_us":33,"thread_start_us":11706,"threads_started":1}
I20260810 14:41:44.758997 15583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:44.739463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37746 (local address 127.21.55.66:33019)
0810 14:41:44.748788 (+ 9325us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:44.748795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:44.748821 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:41:44.748899 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:44.748904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:44.749094 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:44.749302 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:44.751218 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.752192 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.755335 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.756888 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.758364 (+ 1476us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:44.758558 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:44.758640 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 14:41:44.758772 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9159,"thread_start_us":9044,"threads_started":1}
I20260810 14:41:44.758296 15055 tablet_service.cc:1467] Processing CreateTablet for tablet 2fecb5f359164d779cc7cc01fa4806af (DEFAULT_TABLE table=test-table [id=82f044ecb64549e8b4aaf2f9634063aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:44.759732 15577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:44.736265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:36769 (local address 127.0.0.1:55712)
0810 14:41:44.738976 (+ 2711us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:44.738993 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:44.739098 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:44.750647 (+ 11549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:44.750651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:44.750677 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:44.752745 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:44.752766 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.754150 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.754158 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:44.756453 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:44.756469 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.758736 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.758742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:44.758939 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:44.758981 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 14:41:44.759091 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 14:41:44.759229 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2592,"mutex_wait_us":52,"thread_start_us":2460,"threads_started":1}
I20260810 14:41:44.760283 15055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fecb5f359164d779cc7cc01fa4806af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:44.760679 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:44.736862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.21.55.67:36769)
0810 14:41:44.750142 (+ 13280us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:44.750151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:44.750183 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:41:44.750283 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:44.750288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:44.750446 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:44.750658 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:44.752983 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.754003 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.756662 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:44.758132 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:44.758272 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:44.758566 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:44.760093 (+ 1527us) server_negotiation.cc:299] Negotiation successful
0810 14:41:44.760286 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13089,"thread_start_us":12911,"threads_started":1}
I20260810 14:41:44.762975 15259 tablet_service.cc:1467] Processing CreateTablet for tablet 2fecb5f359164d779cc7cc01fa4806af (DEFAULT_TABLE table=test-table [id=82f044ecb64549e8b4aaf2f9634063aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:44.764446 15259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fecb5f359164d779cc7cc01fa4806af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:44.764333 15459 tablet_service.cc:1467] Processing CreateTablet for tablet 2fecb5f359164d779cc7cc01fa4806af (DEFAULT_TABLE table=test-table [id=82f044ecb64549e8b4aaf2f9634063aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:41:44.765681 15459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fecb5f359164d779cc7cc01fa4806af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:41:44.774732 15584 tablet_bootstrap.cc:492] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: Bootstrap starting.
I20260810 14:41:44.777894 15584 tablet_bootstrap.cc:654] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:44.779090 15584 log.cc:824] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:44.780130 15587 tablet_bootstrap.cc:492] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: Bootstrap starting.
I20260810 14:41:44.781127 15585 tablet_bootstrap.cc:492] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: Bootstrap starting.
I20260810 14:41:44.785782 15585 tablet_bootstrap.cc:654] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:44.785970 15587 tablet_bootstrap.cc:654] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: Neither blocks nor log segments found. Creating new log.
I20260810 14:41:44.787216 15585 log.cc:824] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:44.787343 15587 log.cc:824] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: Log is configured to *not* fsync() on all Append() calls
I20260810 14:41:44.789680 15584 tablet_bootstrap.cc:492] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: No bootstrap required, opened a new log
I20260810 14:41:44.790009 15584 ts_tablet_manager.cc:1397] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
I20260810 14:41:44.790292 15585 tablet_bootstrap.cc:492] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: No bootstrap required, opened a new log
I20260810 14:41:44.790612 15585 ts_tablet_manager.cc:1397] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260810 14:41:44.791795 15587 tablet_bootstrap.cc:492] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: No bootstrap required, opened a new log
I20260810 14:41:44.792096 15587 ts_tablet_manager.cc:1397] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260810 14:41:44.800273 15587 raft_consensus.cc:348] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } } peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } }
I20260810 14:41:44.800689 15587 raft_consensus.cc:374] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:44.800792 15587 raft_consensus.cc:729] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a60da52205e49b09d90da0b303ce671, State: Initialized, Role: FOLLOWER
I20260810 14:41:44.801908 15587 consensus_queue.cc:260] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } } peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } }
I20260810 14:41:44.802098 15584 raft_consensus.cc:348] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } } peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } }
I20260810 14:41:44.802103 15585 raft_consensus.cc:348] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } } peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } }
I20260810 14:41:44.802682 15585 raft_consensus.cc:374] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:44.802681 15584 raft_consensus.cc:374] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:41:44.802809 15584 raft_consensus.cc:729] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580e0549c0b241dd8865e230d10963a9, State: Initialized, Role: FOLLOWER
I20260810 14:41:44.802799 15585 raft_consensus.cc:729] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbff42dc4b243f39747ee9030672130, State: Initialized, Role: FOLLOWER
I20260810 14:41:44.803905 15352 heartbeater.cc:503] Master 127.21.55.126:43661 was elected leader, sending a full tablet report...
I20260810 14:41:44.803750 15585 consensus_queue.cc:260] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } } peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } }
I20260810 14:41:44.804032 15584 consensus_queue.cc:260] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } } peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } }
I20260810 14:41:44.805038 15587 ts_tablet_manager.cc:1428] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260810 14:41:44.807385 15550 heartbeater.cc:503] Master 127.21.55.126:43661 was elected leader, sending a full tablet report...
I20260810 14:41:44.808490 15585 ts_tablet_manager.cc:1428] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: Time spent starting tablet: real 0.018s user 0.009s sys 0.007s
I20260810 14:41:44.810081 15584 ts_tablet_manager.cc:1428] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: Time spent starting tablet: real 0.020s user 0.017s sys 0.002s
W20260810 14:41:44.894385 15553 tablet.cc:2369] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:41:44.913179 15158 tablet.cc:2369] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:41:45.052296 15354 tablet.cc:2369] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:41:45.168318 15592 raft_consensus.cc:482] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:41:45.168710 15592 raft_consensus.cc:504] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } } peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } }
I20260810 14:41:45.170687 15592 leader_election.cc:288] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 580e0549c0b241dd8865e230d10963a9 (127.21.55.65:46259), 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769)
I20260810 14:41:45.178880 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:45.171339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:33471 (local address 127.21.55.65:46259)
0810 14:41:45.171569 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:45.171573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:45.172299 (+ 726us) server_negotiation.cc:407] Connection header received
0810 14:41:45.172453 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:45.172460 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:45.172570 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:45.172720 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:45.173666 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.174781 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.176833 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.177708 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.177934 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:45.178014 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:45.178429 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 14:41:45.178604 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 14:41:45.179010 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:45.171339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.21.55.66:33471)
0810 14:41:45.172101 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:45.172121 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:45.172283 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:45.173032 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:45.173040 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:45.173072 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:45.173482 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:45.173492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.174932 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.174939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:45.176634 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:45.176656 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.178042 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.178048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:45.178227 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:45.178259 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:41:45.178366 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:41:45.178499 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":382,"threads_started":1}
I20260810 14:41:45.180154 15597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:45.172173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:36769 (local address 127.21.55.66:43765)
0810 14:41:45.173651 (+ 1478us) negotiation.cc:107] Waiting for socket to connect
0810 14:41:45.173676 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:41:45.173823 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:41:45.174241 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:41:45.174245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:41:45.174273 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:41:45.174788 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:45.174800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.176076 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.176082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:45.177661 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:41:45.177674 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.179351 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.179358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:41:45.179596 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:45.179633 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 14:41:45.179747 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 14:41:45.179890 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1315,"thread_start_us":845,"threads_started":1}
I20260810 14:41:45.180428 15579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:45.172599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:43765 (local address 127.21.55.67:36769)
0810 14:41:45.172911 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:45.172921 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:45.173796 (+ 875us) server_negotiation.cc:407] Connection header received
0810 14:41:45.173969 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:45.173976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:45.174068 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:45.174176 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:41:45.174999 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.175882 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.177851 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.179043 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.179226 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:45.179310 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:45.179877 (+ 567us) server_negotiation.cc:299] Negotiation successful
0810 14:41:45.180082 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260810 14:41:45.180560 15087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fecb5f359164d779cc7cc01fa4806af" candidate_uuid: "2a60da52205e49b09d90da0b303ce671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580e0549c0b241dd8865e230d10963a9" is_pre_election: true
I20260810 14:41:45.181478 15087 raft_consensus.cc:2454] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a60da52205e49b09d90da0b303ce671 in term 0.
I20260810 14:41:45.181754 15489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fecb5f359164d779cc7cc01fa4806af" candidate_uuid: "2a60da52205e49b09d90da0b303ce671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbff42dc4b243f39747ee9030672130" is_pre_election: true
I20260810 14:41:45.182297 15489 raft_consensus.cc:2454] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a60da52205e49b09d90da0b303ce671 in term 0.
I20260810 14:41:45.182346 15192 leader_election.cc:302] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a60da52205e49b09d90da0b303ce671, 580e0549c0b241dd8865e230d10963a9; no voters:
I20260810 14:41:45.182857 15592 raft_consensus.cc:2781] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:41:45.183010 15592 raft_consensus.cc:482] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:41:45.183138 15592 raft_consensus.cc:3037] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:45.185434 15592 raft_consensus.cc:504] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } } peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } }
I20260810 14:41:45.186058 15592 leader_election.cc:288] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [CANDIDATE]: Term 1 election: Requested vote from peers 580e0549c0b241dd8865e230d10963a9 (127.21.55.65:46259), 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769)
I20260810 14:41:45.186640 15087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fecb5f359164d779cc7cc01fa4806af" candidate_uuid: "2a60da52205e49b09d90da0b303ce671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580e0549c0b241dd8865e230d10963a9"
I20260810 14:41:45.186769 15489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fecb5f359164d779cc7cc01fa4806af" candidate_uuid: "2a60da52205e49b09d90da0b303ce671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbff42dc4b243f39747ee9030672130"
I20260810 14:41:45.186885 15087 raft_consensus.cc:3037] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:45.187064 15489 raft_consensus.cc:3037] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:41:45.189221 15087 raft_consensus.cc:2454] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a60da52205e49b09d90da0b303ce671 in term 1.
I20260810 14:41:45.189373 15489 raft_consensus.cc:2454] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a60da52205e49b09d90da0b303ce671 in term 1.
I20260810 14:41:45.189745 15192 leader_election.cc:302] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a60da52205e49b09d90da0b303ce671, 580e0549c0b241dd8865e230d10963a9; no voters:
I20260810 14:41:45.190155 15592 raft_consensus.cc:2781] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:41:45.190714 15592 raft_consensus.cc:686] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 1 LEADER]: Becoming Leader. State: Replica: 2a60da52205e49b09d90da0b303ce671, State: Running, Role: LEADER
I20260810 14:41:45.191279 15592 consensus_queue.cc:237] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } } peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } }
I20260810 14:41:45.196121 14863 catalog_manager.cc:5495] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a60da52205e49b09d90da0b303ce671 (127.21.55.66). New cstate: current_term: 1 leader_uuid: "2a60da52205e49b09d90da0b303ce671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } health_report { overall_health: HEALTHY } } }
I20260810 14:41:45.319975 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:45.310135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37748 (local address 127.21.55.66:33019)
0810 14:41:45.311830 (+ 1695us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:45.311834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:45.311852 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:41:45.311952 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:45.311955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:45.312063 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:45.312278 (+ 215us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:41:45.313204 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.314334 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.316613 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.317132 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.317345 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:45.318073 (+ 728us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:41:45.318924 (+ 851us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:41:45.319084 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:45.319478 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 14:41:45.319594 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1560,"thread_start_us":1305,"threads_started":1}
I20260810 14:41:45.329815 15489 raft_consensus.cc:1261] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130 [term 1 FOLLOWER]: Refusing update from remote peer 2a60da52205e49b09d90da0b303ce671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:41:45.330243 15087 raft_consensus.cc:1261] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Refusing update from remote peer 2a60da52205e49b09d90da0b303ce671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:41:45.331102 15599 consensus_queue.cc:1035] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:41:45.331678 15592 consensus_queue.cc:1035] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:41:45.362361 14860 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:41:45.362775 14860 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:41:45.367239 14860 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Sending DeleteTablet for 3 replicas of tablet 2fecb5f359164d779cc7cc01fa4806af
I20260810 14:41:45.368752 15459 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fecb5f359164d779cc7cc01fa4806af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:45 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:55712
I20260810 14:41:45.368848 15055 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fecb5f359164d779cc7cc01fa4806af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:45 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:50068
I20260810 14:41:45.370365 15626 tablet_replica.cc:330] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: stopping tablet replica
I20260810 14:41:45.370292 15259 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fecb5f359164d779cc7cc01fa4806af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:41:45 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:37746
I20260810 14:41:45.371203 15626 raft_consensus.cc:2229] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:41:45.371567 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 14954
I20260810 14:41:45.372018 15626 raft_consensus.cc:2258] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:45.374595 15626 ts_tablet_manager.cc:1905] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:45.384780 15626 ts_tablet_manager.cc:1918] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:41:45.385134 15626 log.cc:1196] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/2fecb5f359164d779cc7cc01fa4806af
I20260810 14:41:45.385921 15626 ts_tablet_manager.cc:1939] T 2fecb5f359164d779cc7cc01fa4806af P 580e0549c0b241dd8865e230d10963a9: Deleting consensus metadata
I20260810 14:41:45.388702 14846 catalog_manager.cc:4841] TS 580e0549c0b241dd8865e230d10963a9 (127.21.55.65:46259): tablet 2fecb5f359164d779cc7cc01fa4806af (table test-table [id=82f044ecb64549e8b4aaf2f9634063aa]) successfully deleted
I20260810 14:41:45.392333 15627 tablet_replica.cc:330] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: stopping tablet replica
I20260810 14:41:45.393095 15627 raft_consensus.cc:2229] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:41:45.394054 15627 raft_consensus.cc:2258] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:45.396792 15627 ts_tablet_manager.cc:1905] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:45.400792 15624 tablet_replica.cc:330] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: stopping tablet replica
I20260810 14:41:45.401268 15624 raft_consensus.cc:2229] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:41:45.401823 15624 raft_consensus.cc:2258] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:41:45.404776 15627 ts_tablet_manager.cc:1918] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:41:45.404978 15627 log.cc:1196] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/2fecb5f359164d779cc7cc01fa4806af
I20260810 14:41:45.405378 15624 ts_tablet_manager.cc:1905] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:41:45.405601 15627 ts_tablet_manager.cc:1939] T 2fecb5f359164d779cc7cc01fa4806af P 2a60da52205e49b09d90da0b303ce671: Deleting consensus metadata
Aug 10 14:41:45 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:41:45.413600 15624 ts_tablet_manager.cc:1918] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:41:45.413919 15624 log.cc:1196] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/2fecb5f359164d779cc7cc01fa4806af
I20260810 14:41:45.415117 15624 ts_tablet_manager.cc:1939] T 2fecb5f359164d779cc7cc01fa4806af P 9fbff42dc4b243f39747ee9030672130: Deleting consensus metadata
I20260810 14:41:45.417325 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:45.373089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50082 (local address 127.21.55.65:46259)
0810 14:41:45.373265 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:45.373273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:45.391403 (+ 18130us) server_negotiation.cc:407] Connection header received
0810 14:41:45.391654 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:45.391661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:45.391759 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:45.391902 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:45.394163 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.395353 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.398237 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.398925 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.399025 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:45.408828 (+ 9803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:45.408845 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:45.408864 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:45.408927 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:45.413198 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:45.414224 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:45.414235 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:45.414239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:45.414558 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:45.415103 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:45.415108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:45.415112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:45.415610 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:45.415961 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:45.416612 (+ 651us) server_negotiation.cc:299] Negotiation successful
0810 14:41:45.416888 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 14:41:45.418205 14846 catalog_manager.cc:4841] TS 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769): tablet 2fecb5f359164d779cc7cc01fa4806af (table test-table [id=82f044ecb64549e8b4aaf2f9634063aa]) successfully deleted
I20260810 14:41:45.419020 15149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:45.419960 14846 catalog_manager.cc:4841] TS 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019): tablet 2fecb5f359164d779cc7cc01fa4806af (table test-table [id=82f044ecb64549e8b4aaf2f9634063aa]) successfully deleted
I20260810 14:41:45.524266 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 14954
I20260810 14:41:45.559067 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 15166
Aug 10 14:41:45 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:41:45.581358 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:45.560127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37756 (local address 127.21.55.66:33019)
0810 14:41:45.560385 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:45.560391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:45.560412 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:41:45.560620 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:45.560625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:45.560696 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:45.560821 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:45.562375 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.563515 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.565312 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.565963 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.566315 (+ 352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:45.572421 (+ 6106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:45.572458 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:45.572486 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:45.572542 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:45.576474 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:45.578241 (+ 1767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:45.578249 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:45.578252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:45.578709 (+ 457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:45.579436 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:45.579440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:45.579442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:45.579844 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:45.580132 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:45.580692 (+ 560us) server_negotiation.cc:299] Negotiation successful
0810 14:41:45.580933 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 14:41:45.582206 15345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:45.734611 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 15166
I20260810 14:41:45.771888 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 15362
Aug 10 14:41:45 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:41:45.795099 15633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:41:45.773065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.21.55.67:36769)
0810 14:41:45.773766 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0810 14:41:45.773772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:41:45.773794 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:41:45.773865 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:41:45.773869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:41:45.773951 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:41:45.774074 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:41:45.776658 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.777714 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.779794 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:41:45.780301 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:41:45.780440 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:41:45.786671 (+ 6231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:41:45.786700 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:41:45.786725 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:41:45.786800 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:41:45.791098 (+ 4298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:45.791930 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:45.791937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:45.791940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:45.792085 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:41:45.792855 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:41:45.792862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:41:45.792865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:41:45.793586 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:41:45.793849 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 14:41:45.794466 (+ 617us) server_negotiation.cc:299] Negotiation successful
0810 14:41:45.794811 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":420,"threads_started":1}
I20260810 14:41:45.796065 15542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:45.892953 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 15362
I20260810 14:41:45.923440 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 14813
I20260810 14:41:45.925391 14904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:41:46.063230 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 14813
I20260810 14:42:06.097638 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:43661
--webserver_interface=127.21.55.126
--webserver_port=34265
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:43661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:42:06.537425 15638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:06.537842 15638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:06.538024 15638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:06.547317 15638 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 14:42:06.547441 15638 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 14:42:06.547528 15638 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 14:42:06.547642 15638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:42:06.547719 15638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:06.547791 15638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:06.547832 15638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:42:06.547927 15638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:42:06.560984 15638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:43661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:43661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=34265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:06.562479 15638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:06.564034 15638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:06.601682 15647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:06.601639 15650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:06.602908 15648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:06.607511 15638 server_base.cc:1034] running on GCE node
I20260810 14:42:06.610042 15638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:06.613147 15638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:06.614511 15638 hybrid_clock.cc:648] HybridClock initialized: now 1786372926614367 us; error 67 us; skew 500 ppm
I20260810 14:42:06.615372 15638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:06 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372926, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:06.622342 15638 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:42:06.625424 15638 webserver.cc:460] Webserver started at http://127.21.55.126:34265/ using document root <none> and password file <none>
I20260810 14:42:06.627022 15638 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:06.627243 15638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:06.636945 15638 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.000s sys 0.004s
I20260810 14:42:06.642607 15662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:06.645380 15638 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260810 14:42:06.645690 15638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "d0bf2c0428fb4615800a8e628b37861d"
format_stamp: "Formatted at 2026-08-10 14:41:42 on dist-test-slave-h6n0"
server_key: "bf162931067409522fbb674d551c9efd"
server_key_iv: "936bc9fe80a91f7bb36fe495287257b6"
server_key_version: "encryptionkey@0"
I20260810 14:42:06.647764 15638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:06.677609 15638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:06.684499 15638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:06.684824 15638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:06.709903 15638 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:43661
I20260810 14:42:06.709951 15745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:43661 every 8 connection(s)
I20260810 14:42:06.711645 15638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:42:06.712409 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 15638
I20260810 14:42:06.713587 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:46259
--local_ip_for_outbound_sockets=127.21.55.65
--tserver_master_addrs=127.21.55.126:43661
--webserver_port=37441
--webserver_interface=127.21.55.65
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:42:06.721122 15746 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:42:06.724617 15746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Bootstrap starting.
I20260810 14:42:06.745457 15746 log.cc:824] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:06.759593 15746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:42:06.760215 15746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Bootstrap complete.
I20260810 14:42:06.770862 15746 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:42:06.771832 15746 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0bf2c0428fb4615800a8e628b37861d, State: Initialized, Role: FOLLOWER
I20260810 14:42:06.772488 15746 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:42:06.772773 15746 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:42:06.772923 15746 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:42:06.773125 15746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:42:06.777474 15746 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:42:06.778100 15746 leader_election.cc:302] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0bf2c0428fb4615800a8e628b37861d; no voters:
I20260810 14:42:06.778875 15746 leader_election.cc:288] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:42:06.779038 15750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:42:06.780373 15750 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 2 LEADER]: Becoming Leader. State: Replica: d0bf2c0428fb4615800a8e628b37861d, State: Running, Role: LEADER
I20260810 14:42:06.781021 15750 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:42:06.781464 15746 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:42:06.783877 15753 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0bf2c0428fb4615800a8e628b37861d. Latest consensus state: current_term: 2 leader_uuid: "d0bf2c0428fb4615800a8e628b37861d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } } }
I20260810 14:42:06.784703 15753 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: This master's current role is: LEADER
I20260810 14:42:06.785516 15751 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d0bf2c0428fb4615800a8e628b37861d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } } }
I20260810 14:42:06.785823 15751 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: This master's current role is: LEADER
I20260810 14:42:06.797195 15760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:42:06.806232 15760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:42:06.807686 15760 catalog_manager.cc:1259] Loaded cluster ID: 3fcdc3251e3f46a1b7ee2c0a60c5b4e5
I20260810 14:42:06.807845 15760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:42:06.812057 15760 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:42:06.813645 15760 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Loaded TSK: 0
I20260810 14:42:06.825630 15777 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 14:42:06.849915 15760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Generated new TSK 1
I20260810 14:42:06.850371 15760 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:42:07.118587 15748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:07.118921 15748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:07.119088 15748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:07.126881 15748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:07.127014 15748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:07.127183 15748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:42:07.140220 15748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:46259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=37441
--tserver_master_addrs=127.21.55.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:07.141834 15748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:07.143963 15748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:07.162660 15787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:07.163369 15790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:07.163937 15786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:07.169005 15748 server_base.cc:1034] running on GCE node
I20260810 14:42:07.169813 15748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:07.171512 15748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:07.172919 15748 hybrid_clock.cc:648] HybridClock initialized: now 1786372927172848 us; error 76 us; skew 500 ppm
I20260810 14:42:07.173429 15748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:07 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372927, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:07.178625 15748 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:42:07.181082 15748 webserver.cc:460] Webserver started at http://127.21.55.65:37441/ using document root <none> and password file <none>
I20260810 14:42:07.182686 15748 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:07.182958 15748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:07.191438 15748 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.000s sys 0.004s
I20260810 14:42:07.194869 15800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:07.196429 15748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:42:07.196621 15748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "580e0549c0b241dd8865e230d10963a9"
format_stamp: "Formatted at 2026-08-10 14:41:43 on dist-test-slave-h6n0"
server_key: "5c107c5a5ddd9780191ffc9d6a33adef"
server_key_iv: "5771bec2725c739cc1e4e193b49968f2"
server_key_version: "encryptionkey@0"
I20260810 14:42:07.197697 15748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:07.217744 15748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:07.226441 15748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:07.226797 15748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:07.228102 15748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:07.229902 15748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:07.230012 15748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:07.230146 15748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:07.230238 15748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:07.278170 15748 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:46259
I20260810 14:42:07.278250 15966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:46259 every 8 connection(s)
I20260810 14:42:07.280014 15748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:42:07.281901 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 15748
I20260810 14:42:07.283365 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:33019
--local_ip_for_outbound_sockets=127.21.55.66
--tserver_master_addrs=127.21.55.126:43661
--webserver_port=37665
--webserver_interface=127.21.55.66
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:42:07 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372927, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:42:07.313277 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:07.285605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:35979)
0810 14:42:07.286868 (+ 1263us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:07.286938 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:07.289144 (+ 2206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:07.290857 (+ 1713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:07.290868 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:07.291665 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:07.292449 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:07.292477 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:07.294992 (+ 2515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:07.295000 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:07.297164 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:07.297184 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:07.299038 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:07.299052 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:07.299473 (+ 421us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:07.301089 (+ 1616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:42:07.301151 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:07.304576 (+ 3425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:07.308419 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:07.308431 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:07.308448 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:07.308951 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:07.309453 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:07.309458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:07.309462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:07.309667 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:07.310502 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:07.310510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:07.310912 (+ 402us) client_negotiation.cc:769] Sending connection context
0810 14:42:07.311201 (+ 289us) client_negotiation.cc:240] Negotiation successful
0810 14:42:07.311846 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":501,"threads_started":1}
I20260810 14:42:07.313268 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:07.285025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:35979 (local address 127.21.55.126:43661)
0810 14:42:07.285785 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:07.285800 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:07.287104 (+ 1304us) server_negotiation.cc:407] Connection header received
0810 14:42:07.289824 (+ 2720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:07.289837 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:07.290230 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:07.290677 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:07.292726 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:07.294779 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:07.297493 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:07.298693 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:07.298898 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:07.304925 (+ 6027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:07.304934 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:07.304950 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:07.305001 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:07.308210 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:07.309128 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:07.309135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:07.309141 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:07.309307 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:07.309798 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:07.309808 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:07.309810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:07.310327 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:07.310569 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:07.311446 (+ 877us) server_negotiation.cc:299] Negotiation successful
0810 14:42:07.311846 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":389,"threads_started":1}
I20260810 14:42:07.318385 15967 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43661
I20260810 14:42:07.318938 15967 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:07.320096 15967 heartbeater.cc:511] Master 127.21.55.126:43661 requested a full tablet report, sending...
I20260810 14:42:07.324368 15689 ts_manager.cc:194] Registered new tserver with Master: 580e0549c0b241dd8865e230d10963a9 (127.21.55.65:46259)
I20260810 14:42:07.326828 15689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:35979
W20260810 14:42:07.709514 15972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:07.710150 15972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:07.710707 15972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:07.719478 15972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:07.719660 15972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:07.719857 15972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:42:07.730420 15972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:33019
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=37665
--tserver_master_addrs=127.21.55.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:07.731897 15972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:07.733349 15972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:07.748487 15981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:07.748581 15983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:07.748607 15980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:07.749755 15972 server_base.cc:1034] running on GCE node
I20260810 14:42:07.750391 15972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:07.751691 15972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:07.753218 15972 hybrid_clock.cc:648] HybridClock initialized: now 1786372927753132 us; error 69 us; skew 500 ppm
I20260810 14:42:07.753577 15972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:07 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372927, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:07.757578 15972 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:42:07.759675 15972 webserver.cc:460] Webserver started at http://127.21.55.66:37665/ using document root <none> and password file <none>
I20260810 14:42:07.760439 15972 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:07.760632 15972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:07.767100 15972 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:42:07.770243 15993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:07.771787 15972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:42:07.772017 15972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "2a60da52205e49b09d90da0b303ce671"
format_stamp: "Formatted at 2026-08-10 14:41:43 on dist-test-slave-h6n0"
server_key: "9ba922d800e835b89376f3b9fc8feb4e"
server_key_iv: "2c2fe987edbf123f089aa0d33f918113"
server_key_version: "encryptionkey@0"
I20260810 14:42:07.773188 15972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:07.786087 15972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:07.791512 15972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:07.791889 15972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:07.793164 15972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:07.795086 15972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:07.795244 15972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:07.795395 15972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:07.795492 15972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:07.839008 15972 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:33019
I20260810 14:42:07.839103 16171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:33019 every 8 connection(s)
I20260810 14:42:07.840538 15972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:42:07.845794 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 15972
I20260810 14:42:07.846510 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:36769
--local_ip_for_outbound_sockets=127.21.55.67
--tserver_master_addrs=127.21.55.126:43661
--webserver_port=35107
--webserver_interface=127.21.55.67
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:42:07 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372927, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:42:07.865847 16175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:07.844615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:42645 (local address 127.21.55.126:43661)
0810 14:42:07.845242 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:07.845247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:07.846279 (+ 1032us) server_negotiation.cc:407] Connection header received
0810 14:42:07.848519 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:07.848528 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:07.848641 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:07.848777 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:07.850431 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:07.851549 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:07.853350 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:07.854091 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:07.854239 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:07.859020 (+ 4781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:07.859025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:07.859028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:07.859062 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:07.861787 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:07.862906 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:07.862915 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:07.862919 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:07.863090 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:07.863665 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:07.863671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:07.863674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:07.864023 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:07.864216 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:07.865061 (+ 845us) server_negotiation.cc:299] Negotiation successful
0810 14:42:07.865352 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":394,"threads_started":1}
I20260810 14:42:07.867030 16176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:07.845136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:42645)
0810 14:42:07.846087 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:07.846147 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:07.848227 (+ 2080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:07.849025 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:07.849033 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:07.849610 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:07.850239 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:07.850257 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:07.851761 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:07.851766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:07.853182 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:07.853196 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:07.854385 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:07.854392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:07.854705 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:07.855548 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:42:07.855603 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:07.858783 (+ 3180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:07.862077 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:07.862090 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:07.862109 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:07.862747 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:07.863238 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:07.863245 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:07.863249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:07.863502 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:07.864192 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:07.864202 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:07.864593 (+ 391us) client_negotiation.cc:769] Sending connection context
0810 14:42:07.864942 (+ 349us) client_negotiation.cc:240] Negotiation successful
0810 14:42:07.865564 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":408,"threads_started":1}
I20260810 14:42:07.869184 16173 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43661
I20260810 14:42:07.869572 16173 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:07.870494 16173 heartbeater.cc:511] Master 127.21.55.126:43661 requested a full tablet report, sending...
I20260810 14:42:07.872349 15689 ts_manager.cc:194] Registered new tserver with Master: 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019)
I20260810 14:42:07.873481 15689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:42645
W20260810 14:42:08.237211 16177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:08.237620 16177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:08.237802 16177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:08.248136 16177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:08.248346 16177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:08.248775 16177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:42:08.264169 16177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:36769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=35107
--tserver_master_addrs=127.21.55.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:08.265695 16177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:08.268792 16177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:08.296669 16188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:08.296686 16186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:08.303495 16191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:08.302728 16177 server_base.cc:1034] running on GCE node
I20260810 14:42:08.305260 16177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:08.306974 16177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:08.311169 16177 hybrid_clock.cc:648] HybridClock initialized: now 1786372928310391 us; error 192 us; skew 500 ppm
I20260810 14:42:08.311764 16177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:08 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372928, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:08.322440 16177 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:42:08.325917 16177 webserver.cc:460] Webserver started at http://127.21.55.67:35107/ using document root <none> and password file <none>
I20260810 14:42:08.326970 16177 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:08.327185 16177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:08.332520 15967 heartbeater.cc:503] Master 127.21.55.126:43661 was elected leader, sending a full tablet report...
I20260810 14:42:08.336692 16177 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.004s sys 0.004s
I20260810 14:42:08.341032 16202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:08.343783 16177 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260810 14:42:08.344046 16177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "9fbff42dc4b243f39747ee9030672130"
format_stamp: "Formatted at 2026-08-10 14:41:44 on dist-test-slave-h6n0"
server_key: "1aa03bacc9924dc835558d331b4db2d0"
server_key_iv: "fe7d746c6231f4ef3ada432848ddbecb"
server_key_version: "encryptionkey@0"
I20260810 14:42:08.345331 16177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:08.369293 16177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:08.375061 16177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:08.375417 16177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:08.377143 16177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:08.378877 16177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:08.379019 16177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:08.379156 16177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:08.379248 16177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:08.429103 16177 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:36769
I20260810 14:42:08.429153 16372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:36769 every 8 connection(s)
I20260810 14:42:08.430716 16177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:42:08.440157 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 16177
Aug 10 14:42:08 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372928, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:42:08.465101 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.434042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:54385 (local address 127.21.55.126:43661)
0810 14:42:08.434943 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.434949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.435638 (+ 689us) server_negotiation.cc:407] Connection header received
0810 14:42:08.439972 (+ 4334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.439983 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.440166 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.440373 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:08.445315 (+ 4942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.446656 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.449600 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.450332 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.450440 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.457072 (+ 6632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:08.457086 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:08.457093 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:08.457165 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:08.460920 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:08.461929 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:08.461933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:08.461936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:08.462074 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:08.462656 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:08.462660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:08.462662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:08.463037 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:08.463220 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.464262 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.464589 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"spinlock_wait_cycles":4352,"thread_start_us":536,"threads_started":1}
I20260810 14:42:08.465369 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.441582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.21.55.126:43661)
0810 14:42:08.443403 (+ 1821us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.443412 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.443439 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:42:08.443533 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.443538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.443637 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.443788 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:08.446492 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.448252 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.450230 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.451543 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.451676 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.456110 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:08.456117 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:08.456121 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:08.456259 (+ 138us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:08.460403 (+ 4144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:08.461850 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:08.461857 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:08.461864 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:08.462071 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:08.462802 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:08.462807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:08.462809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:08.463168 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:08.463798 (+ 630us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.464331 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.464860 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":168,"server-negotiator.queue_time_us":1492,"thread_start_us":1291,"threads_started":1}
I20260810 14:42:08.467485 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.434377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:54385)
0810 14:42:08.435373 (+ 996us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:08.435441 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:08.438146 (+ 2705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:08.441443 (+ 3297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:08.441464 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:08.443704 (+ 2240us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:08.444801 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.444825 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.447180 (+ 2355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.447188 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.449388 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.449413 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.450664 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.450675 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.451598 (+ 923us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.452656 (+ 1058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:42:08.452686 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:08.456548 (+ 3862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:08.461116 (+ 4568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:08.461130 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:08.461162 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:08.461746 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:08.462202 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:08.462207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:08.462211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:08.462469 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:08.463187 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:08.463197 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:08.463671 (+ 474us) client_negotiation.cc:769] Sending connection context
0810 14:42:08.464150 (+ 479us) client_negotiation.cc:240] Negotiation successful
0810 14:42:08.465236 (+ 1086us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":405,"threads_started":1}
I20260810 14:42:08.469437 16376 heartbeater.cc:347] Connected to a master server at 127.21.55.126:43661
I20260810 14:42:08.469954 16376 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:08.471464 16376 heartbeater.cc:511] Master 127.21.55.126:43661 requested a full tablet report, sending...
I20260810 14:42:08.473709 15689 ts_manager.cc:194] Registered new tserver with Master: 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769)
I20260810 14:42:08.474727 15689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:54385
I20260810 14:42:08.483129 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:42:08.493403 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.484517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.21.55.126:43661)
0810 14:42:08.484883 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.484888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.485712 (+ 824us) server_negotiation.cc:407] Connection header received
0810 14:42:08.486018 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.486023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.486220 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.486528 (+ 308us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:42:08.487235 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.488977 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.490945 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.491494 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.491622 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.492425 (+ 803us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:42:08.492746 (+ 321us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 14:42:08.493153 (+ 407us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41226: token expired
Metrics: {"server-negotiator.queue_time_us":169}
W20260810 14:42:08.493518 16384 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41226: token expired
W20260810 14:42:08.495158 15556 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.55.126:43661)
I20260810 14:42:08.512255 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.495800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.21.55.126:43661)
0810 14:42:08.496029 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.496038 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.496059 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:42:08.496254 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.496261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.496331 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.496437 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:08.498615 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.499458 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.501014 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.501526 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.501669 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.505472 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:08.505487 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:08.505491 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:08.505530 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:08.508855 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:08.509781 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:08.509785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:08.509789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:08.509963 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:08.510403 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:08.510408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:08.510410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:08.511118 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:08.511329 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.511717 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.511896 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:42:08.516160 15689 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:42:08.524502 15689 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:42:08.557471 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.543459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:33019 (local address 127.0.0.1:33614)
0810 14:42:08.549532 (+ 6073us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:08.549554 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:08.549711 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:08.550570 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:08.550578 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:08.550605 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:08.551061 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.551072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.552850 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.552856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.554697 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.554709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.556461 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.556468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.556692 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.556753 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 14:42:08.556885 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 14:42:08.557089 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5956,"thread_start_us":5651,"threads_started":1}
I20260810 14:42:08.558701 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.544111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33614 (local address 127.21.55.66:33019)
0810 14:42:08.544940 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.544950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.549694 (+ 4744us) server_negotiation.cc:407] Connection header received
0810 14:42:08.549919 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.549928 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.550242 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.551279 (+ 1037us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:08.551581 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.552699 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.554864 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.556209 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.556387 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.556616 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.557762 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.557959 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":450,"threads_started":1}
I20260810 14:42:08.560218 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.541663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:36769 (local address 127.0.0.1:59592)
0810 14:42:08.542648 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:08.542704 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:08.542849 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:08.544787 (+ 1938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:08.544800 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:08.544846 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:08.545658 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.545678 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.547774 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.547787 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.556605 (+ 8818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.556619 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.559389 (+ 2770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.559396 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.559597 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.559632 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 14:42:08.559710 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 14:42:08.559843 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":927,"threads_started":1}
I20260810 14:42:08.560382 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.542348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59592 (local address 127.21.55.67:36769)
0810 14:42:08.543517 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.543528 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.543584 (+ 56us) server_negotiation.cc:407] Connection header received
0810 14:42:08.543701 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.543710 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.544022 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.544293 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:08.545980 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.547483 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.556786 (+ 9303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.558882 (+ 2096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.559263 (+ 381us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.559582 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.559796 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.560011 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":977,"thread_start_us":1258,"threads_started":1}
I20260810 14:42:08.563903 16277 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa509261fe74a758decf59406b11ca5 (DEFAULT_TABLE table=test-table [id=f14be318799d481b94209b3d444c48a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:08.563917 16063 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa509261fe74a758decf59406b11ca5 (DEFAULT_TABLE table=test-table [id=f14be318799d481b94209b3d444c48a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:08.565379 16063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa509261fe74a758decf59406b11ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:08.565384 16277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa509261fe74a758decf59406b11ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:08.567322 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.549909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.0.0.1:33254)
0810 14:42:08.550596 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:08.550620 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:08.550734 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:08.560206 (+ 9472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:08.560212 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:08.560242 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:08.560577 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.560589 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.562229 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.562234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.564235 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.564250 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.566477 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.566484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.566701 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.566735 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 14:42:08.566838 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 14:42:08.566977 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":310,"threads_started":1}
I20260810 14:42:08.568109 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.550007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.21.55.65:46259)
0810 14:42:08.558893 (+ 8886us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.558904 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.558947 (+ 43us) server_negotiation.cc:407] Connection header received
0810 14:42:08.559044 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.559052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.559329 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.559945 (+ 616us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:08.560757 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.562077 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.564417 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.565905 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.566059 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.566296 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.567576 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.567738 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8668,"thread_start_us":8522,"threads_started":1}
I20260810 14:42:08.572386 15878 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa509261fe74a758decf59406b11ca5 (DEFAULT_TABLE table=test-table [id=f14be318799d481b94209b3d444c48a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:08.574064 15878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa509261fe74a758decf59406b11ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:08.579280 16403 tablet_bootstrap.cc:492] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: Bootstrap starting.
I20260810 14:42:08.583076 16403 tablet_bootstrap.cc:654] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:08.584216 16403 log.cc:824] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:08.591902 16406 tablet_bootstrap.cc:492] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671: Bootstrap starting.
I20260810 14:42:08.593780 16410 tablet_bootstrap.cc:492] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Bootstrap starting.
I20260810 14:42:08.598376 16403 tablet_bootstrap.cc:492] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: No bootstrap required, opened a new log
I20260810 14:42:08.598673 16406 tablet_bootstrap.cc:654] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:08.598825 16403 ts_tablet_manager.cc:1397] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.013s
I20260810 14:42:08.598877 16410 tablet_bootstrap.cc:654] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:08.599913 16406 log.cc:824] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:08.600215 16410 log.cc:824] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:08.603332 16410 tablet_bootstrap.cc:492] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: No bootstrap required, opened a new log
I20260810 14:42:08.603511 16406 tablet_bootstrap.cc:492] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671: No bootstrap required, opened a new log
I20260810 14:42:08.603636 16410 ts_tablet_manager.cc:1397] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.001s
I20260810 14:42:08.603820 16406 ts_tablet_manager.cc:1397] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260810 14:42:08.611833 16403 raft_consensus.cc:348] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:08.612713 16403 raft_consensus.cc:374] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:08.612861 16403 raft_consensus.cc:729] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbff42dc4b243f39747ee9030672130, State: Initialized, Role: FOLLOWER
I20260810 14:42:08.612471 16410 raft_consensus.cc:348] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:08.613092 16410 raft_consensus.cc:374] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:08.613230 16410 raft_consensus.cc:729] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580e0549c0b241dd8865e230d10963a9, State: Initialized, Role: FOLLOWER
I20260810 14:42:08.613741 16403 consensus_queue.cc:260] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:08.614256 16410 consensus_queue.cc:260] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:08.615259 16376 heartbeater.cc:503] Master 127.21.55.126:43661 was elected leader, sending a full tablet report...
I20260810 14:42:08.615899 16403 ts_tablet_manager.cc:1428] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: Time spent starting tablet: real 0.017s user 0.012s sys 0.003s
I20260810 14:42:08.616602 16406 raft_consensus.cc:348] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:08.617121 16406 raft_consensus.cc:374] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:08.617228 16406 raft_consensus.cc:729] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a60da52205e49b09d90da0b303ce671, State: Initialized, Role: FOLLOWER
I20260810 14:42:08.617738 16410 ts_tablet_manager.cc:1428] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260810 14:42:08.617879 16406 consensus_queue.cc:260] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:08.620054 16173 heartbeater.cc:503] Master 127.21.55.126:43661 was elected leader, sending a full tablet report...
I20260810 14:42:08.621048 16406 ts_tablet_manager.cc:1428] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260810 14:42:08.649117 16413 raft_consensus.cc:482] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:42:08.649821 16413 raft_consensus.cc:504] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:08.651335 16414 raft_consensus.cc:482] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:42:08.651723 16414 raft_consensus.cc:504] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:08.652592 16413 leader_election.cc:288] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019), 580e0549c0b241dd8865e230d10963a9 (127.21.55.65:46259)
I20260810 14:42:08.654001 16414 leader_election.cc:288] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769), 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019)
I20260810 14:42:08.662810 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.653073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:36875 (local address 127.21.55.66:33019)
0810 14:42:08.653417 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.653425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.653573 (+ 148us) server_negotiation.cc:407] Connection header received
0810 14:42:08.653908 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.653915 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.654016 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.654131 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:08.654870 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.656077 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.659682 (+ 3605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.661134 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.661295 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.661377 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.662334 (+ 957us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.662484 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260810 14:42:08.662797 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.653088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:33019 (local address 127.21.55.67:36875)
0810 14:42:08.653417 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:08.653446 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:08.653649 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:08.654336 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:08.654340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:08.654363 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:08.654642 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.654657 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.656261 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.656270 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.658738 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.658762 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.661900 (+ 3138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.661909 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.662123 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.662164 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 14:42:08.662304 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 14:42:08.662453 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260810 14:42:08.665547 16416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.653680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.21.55.67:37083)
0810 14:42:08.655172 (+ 1492us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:08.655192 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:08.655413 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:08.656048 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:08.656053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:08.656079 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:08.656561 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.656582 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.658332 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.658338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.660781 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.660797 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.664529 (+ 3732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.664535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.664717 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.664749 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:42:08.664847 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 14:42:08.665014 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1343,"mutex_wait_us":1,"thread_start_us":1146,"threads_started":1}
I20260810 14:42:08.667466 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.654077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:37083 (local address 127.21.55.65:46259)
0810 14:42:08.654879 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.654887 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.655288 (+ 401us) server_negotiation.cc:407] Connection header received
0810 14:42:08.655666 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.655673 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.655757 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.655820 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:08.656810 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.658131 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.661027 (+ 2896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.662678 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.662882 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.662977 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.666839 (+ 3862us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.667046 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":584}
I20260810 14:42:08.668727 15901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa509261fe74a758decf59406b11ca5" candidate_uuid: "9fbff42dc4b243f39747ee9030672130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580e0549c0b241dd8865e230d10963a9" is_pre_election: true
I20260810 14:42:08.669232 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.655256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:54851 (local address 127.21.55.67:36769)
0810 14:42:08.655522 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.655532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.656512 (+ 980us) server_negotiation.cc:407] Connection header received
0810 14:42:08.656886 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.656897 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.656991 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.657075 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:08.658447 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.660174 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.664779 (+ 4605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.667769 (+ 2990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.667925 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.668008 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.668814 (+ 806us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.668968 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":84}
I20260810 14:42:08.669296 15901 raft_consensus.cc:2454] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbff42dc4b243f39747ee9030672130 in term 0.
I20260810 14:42:08.670336 16209 leader_election.cc:302] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 580e0549c0b241dd8865e230d10963a9, 9fbff42dc4b243f39747ee9030672130; no voters:
I20260810 14:42:08.670873 16413 raft_consensus.cc:2781] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:42:08.671072 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.655255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:36769 (local address 127.21.55.65:54851)
0810 14:42:08.656378 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:08.656401 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:08.656672 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:08.657372 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:08.657381 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:08.657416 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:08.658151 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.658178 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.660408 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.660429 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.664528 (+ 4099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.664551 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.668276 (+ 3725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.668289 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.668575 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.668618 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 14:42:08.669475 (+ 857us) client_negotiation.cc:240] Negotiation successful
0810 14:42:08.669646 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"mutex_wait_us":2,"thread_start_us":528,"threads_started":1}
I20260810 14:42:08.671298 16413 raft_consensus.cc:482] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:42:08.671512 16413 raft_consensus.cc:3037] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:08.672245 16302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa509261fe74a758decf59406b11ca5" candidate_uuid: "580e0549c0b241dd8865e230d10963a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbff42dc4b243f39747ee9030672130" is_pre_election: true
I20260810 14:42:08.673070 16093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa509261fe74a758decf59406b11ca5" candidate_uuid: "9fbff42dc4b243f39747ee9030672130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a60da52205e49b09d90da0b303ce671" is_pre_election: true
I20260810 14:42:08.673728 16093 raft_consensus.cc:2454] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbff42dc4b243f39747ee9030672130 in term 0.
I20260810 14:42:08.675540 16413 raft_consensus.cc:504] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:08.676352 16413 leader_election.cc:288] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [CANDIDATE]: Term 1 election: Requested vote from peers 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019), 580e0549c0b241dd8865e230d10963a9 (127.21.55.65:46259)
I20260810 14:42:08.677115 16093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa509261fe74a758decf59406b11ca5" candidate_uuid: "9fbff42dc4b243f39747ee9030672130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a60da52205e49b09d90da0b303ce671"
I20260810 14:42:08.677398 16093 raft_consensus.cc:3037] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:08.677477 15901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa509261fe74a758decf59406b11ca5" candidate_uuid: "9fbff42dc4b243f39747ee9030672130" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "580e0549c0b241dd8865e230d10963a9"
I20260810 14:42:08.677686 15901 raft_consensus.cc:3037] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:08.678061 16302 raft_consensus.cc:2379] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 580e0549c0b241dd8865e230d10963a9 in current term 1: Already voted for candidate 9fbff42dc4b243f39747ee9030672130 in this term.
I20260810 14:42:08.680389 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.657028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:59583 (local address 127.21.55.66:33019)
0810 14:42:08.663929 (+ 6901us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.663938 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.663958 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:42:08.664049 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.664054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.664207 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.664298 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:08.666605 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.668002 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.671286 (+ 3284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.673509 (+ 2223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.673699 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.673781 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.679844 (+ 6063us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.680048 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6009,"thread_start_us":15103,"threads_started":1}
I20260810 14:42:08.681172 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.656369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:33019 (local address 127.21.55.65:59583)
0810 14:42:08.663510 (+ 7141us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:08.663541 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:08.663681 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:08.665807 (+ 2126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:08.665816 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:08.665853 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:08.666347 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.666362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.668264 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.668272 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.671075 (+ 2803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:08.671095 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.679361 (+ 8266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.679369 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:08.679597 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.679643 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 14:42:08.680591 (+ 948us) client_negotiation.cc:240] Negotiation successful
0810 14:42:08.680756 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6978,"thread_start_us":7093,"threads_started":1}
I20260810 14:42:08.682368 16093 raft_consensus.cc:2454] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbff42dc4b243f39747ee9030672130 in term 1.
W20260810 14:42:08.682636 16377 tablet.cc:2369] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:42:08.682579 16091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa509261fe74a758decf59406b11ca5" candidate_uuid: "580e0549c0b241dd8865e230d10963a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a60da52205e49b09d90da0b303ce671" is_pre_election: true
I20260810 14:42:08.683050 16091 raft_consensus.cc:2379] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 580e0549c0b241dd8865e230d10963a9 in current term 1: Already voted for candidate 9fbff42dc4b243f39747ee9030672130 in this term.
I20260810 14:42:08.682996 16209 leader_election.cc:302] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a60da52205e49b09d90da0b303ce671, 9fbff42dc4b243f39747ee9030672130; no voters:
I20260810 14:42:08.683698 16413 raft_consensus.cc:2781] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:42:08.683768 15804 leader_election.cc:302] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 580e0549c0b241dd8865e230d10963a9; no voters: 2a60da52205e49b09d90da0b303ce671, 9fbff42dc4b243f39747ee9030672130
I20260810 14:42:08.684424 15901 raft_consensus.cc:2454] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbff42dc4b243f39747ee9030672130 in term 1.
I20260810 14:42:08.684626 16414 raft_consensus.cc:2726] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 14:42:08.685881 16413 raft_consensus.cc:686] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 1 LEADER]: Becoming Leader. State: Replica: 9fbff42dc4b243f39747ee9030672130, State: Running, Role: LEADER
I20260810 14:42:08.686466 16413 consensus_queue.cc:237] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:08.691771 15688 catalog_manager.cc:5495] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fbff42dc4b243f39747ee9030672130 (127.21.55.67). New cstate: current_term: 1 leader_uuid: "9fbff42dc4b243f39747ee9030672130" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } health_report { overall_health: UNKNOWN } } }
I20260810 14:42:08.733870 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.727440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59598 (local address 127.21.55.67:36769)
0810 14:42:08.727640 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.727644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.727686 (+ 42us) server_negotiation.cc:407] Connection header received
0810 14:42:08.727903 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.727907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.728005 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.728166 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:42:08.728903 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.729770 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.731596 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.732036 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.732145 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.732725 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:42:08.733247 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:42:08.733365 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.733526 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.733624 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 14:42:08.742668 15901 raft_consensus.cc:1261] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Refusing update from remote peer 9fbff42dc4b243f39747ee9030672130: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:42:08.742748 16091 raft_consensus.cc:1261] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671 [term 1 FOLLOWER]: Refusing update from remote peer 9fbff42dc4b243f39747ee9030672130: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:42:08.743685 16422 consensus_queue.cc:1035] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [LEADER]: Connected to new peer: Peer: permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:42:08.744131 16413 consensus_queue.cc:1035] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:42:08.775513 15688 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:42:08.775996 15688 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:42:08.781773 15688 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Sending DeleteTablet for 3 replicas of tablet 2aa509261fe74a758decf59406b11ca5
I20260810 14:42:08.783237 16277 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa509261fe74a758decf59406b11ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:08 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:59592
I20260810 14:42:08.783381 15878 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa509261fe74a758decf59406b11ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:08 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:33254
I20260810 14:42:08.783718 16063 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa509261fe74a758decf59406b11ca5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:08 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:33614
I20260810 14:42:08.784755 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 15748
I20260810 14:42:08.795001 16439 tablet_replica.cc:330] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: stopping tablet replica
I20260810 14:42:08.795648 16439 raft_consensus.cc:2229] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:42:08.796568 16439 raft_consensus.cc:2258] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:42:08.796644 16440 tablet_replica.cc:330] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: stopping tablet replica
I20260810 14:42:08.797426 16443 tablet_replica.cc:330] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671: stopping tablet replica
I20260810 14:42:08.797660 16440 raft_consensus.cc:2229] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:42:08.799021 16443 raft_consensus.cc:2229] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:42:08.799546 16440 raft_consensus.cc:2258] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:42:08.800534 16443 raft_consensus.cc:2258] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:42:08.803980 16439 ts_tablet_manager.cc:1905] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:08.804575 16443 ts_tablet_manager.cc:1905] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:08.812068 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.788239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33266 (local address 127.21.55.65:46259)
0810 14:42:08.788450 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.788456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.788478 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:42:08.788547 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.788553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.788639 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.788748 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:08.791148 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.793382 (+ 2234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.795972 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.796816 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.796926 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.803207 (+ 6281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:08.803226 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:08.803247 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:08.803303 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:08.807480 (+ 4177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:08.808527 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:08.808536 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:08.808540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:08.808720 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:08.809229 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:08.809234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:08.809238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:08.809722 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:08.810081 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.811388 (+ 1307us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.811669 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 14:42:08.812106 16440 ts_tablet_manager.cc:1905] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:08.812808 16439 ts_tablet_manager.cc:1918] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:42:08.813014 16439 log.cc:1196] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/2aa509261fe74a758decf59406b11ca5
I20260810 14:42:08.813032 15960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:08.813769 16439 ts_tablet_manager.cc:1939] T 2aa509261fe74a758decf59406b11ca5 P 9fbff42dc4b243f39747ee9030672130: Deleting consensus metadata
I20260810 14:42:08.814101 16443 ts_tablet_manager.cc:1918] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:42:08.814283 16443 log.cc:1196] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/2aa509261fe74a758decf59406b11ca5
I20260810 14:42:08.814899 16443 ts_tablet_manager.cc:1939] T 2aa509261fe74a758decf59406b11ca5 P 2a60da52205e49b09d90da0b303ce671: Deleting consensus metadata
I20260810 14:42:08.816267 15666 catalog_manager.cc:4841] TS 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769): tablet 2aa509261fe74a758decf59406b11ca5 (table test-table [id=f14be318799d481b94209b3d444c48a4]) successfully deleted
I20260810 14:42:08.816725 15666 catalog_manager.cc:4841] TS 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019): tablet 2aa509261fe74a758decf59406b11ca5 (table test-table [id=f14be318799d481b94209b3d444c48a4]) successfully deleted
I20260810 14:42:08.949975 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 15748
W20260810 14:42:08.979627 15666 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.55.65:46259 (error 108)
I20260810 14:42:08.979878 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 15972
I20260810 14:42:08.982183 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.981637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.0.0.1:37016)
0810 14:42:08.981854 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:08.981992 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 14:42:08.982473 15666 catalog_manager.cc:4584] TS 580e0549c0b241dd8865e230d10963a9 (127.21.55.65:46259): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2aa509261fe74a758decf59406b11ca5: Network error: Client connection negotiation failed: client connection to 127.21.55.65:46259: connect: Connection refused (error 111)
I20260810 14:42:08.998667 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:08.980809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33104 (local address 127.21.55.66:33019)
0810 14:42:08.981015 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:08.981022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:08.981047 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:42:08.981294 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:08.981300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:08.981402 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:08.981533 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:08.983567 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.984823 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.987131 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:08.987851 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:08.988002 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:08.990816 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:08.990835 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:08.990856 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:08.990899 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:08.995148 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:08.996120 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:08.996131 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:08.996135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:08.996313 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:08.996784 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:08.996790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:08.996794 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:08.997287 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:08.997708 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:08.997928 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 14:42:08.998204 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:42:08.999629 16164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:09.036350 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:09.035763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.0.0.1:37032)
0810 14:42:09.036014 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:09.036151 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 14:42:09.037459 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:09.037094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.0.0.1:37044)
0810 14:42:09.037247 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:09.037329 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 14:42:09.098655 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:09.098038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.0.0.1:37054)
0810 14:42:09.098300 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:09.098382 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260810 14:42:09.099748 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 15972
I20260810 14:42:09.100209 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:09.099578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.0.0.1:37068)
0810 14:42:09.099928 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:09.100053 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195}
I20260810 14:42:09.130334 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 16177
I20260810 14:42:09.152472 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:09.131694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45832 (local address 127.21.55.67:36769)
0810 14:42:09.131989 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:09.131995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:09.132025 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:42:09.132254 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:09.132259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:09.132351 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:09.132452 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:09.134476 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:09.135368 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:09.137631 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:09.138121 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:09.138227 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:09.142365 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:09.142397 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:09.142420 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:09.142505 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:09.147918 (+ 5413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:09.149326 (+ 1408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:09.149337 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:09.149340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:09.149591 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:09.150196 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:09.150200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:09.150202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:09.150734 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:09.151339 (+ 605us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:09.151779 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 14:42:09.152039 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260810 14:42:09.153726 16367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:09.190614 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:09.189966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.0.0.1:37082)
0810 14:42:09.190222 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:09.190329 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260810 14:42:09.191974 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:09.191468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.0.0.1:37092)
0810 14:42:09.191731 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:09.191833 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260810 14:42:09.296427 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 16177
I20260810 14:42:09.329008 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 15638
I20260810 14:42:09.330320 15738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:09.465358 16398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:09.464801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:46259 (local address 127.0.0.1:37098)
0810 14:42:09.465004 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:09.465124 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 14:42:09.465467 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 15638
I20260810 14:42:29.498158 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:43661
--webserver_interface=127.21.55.126
--webserver_port=34265
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:43661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:42:29.857735 16449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:29.858011 16449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:29.858103 16449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:29.865944 16449 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 14:42:29.866041 16449 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 14:42:29.866079 16449 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 14:42:29.866123 16449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:42:29.866175 16449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:29.866241 16449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:29.866285 16449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:42:29.866344 16449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:42:29.875965 16449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:43661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:43661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=34265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:29.877100 16449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:29.878350 16449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:29.890693 16456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:29.890655 16457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:29.890733 16460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:29.892246 16449 server_base.cc:1034] running on GCE node
I20260810 14:42:29.893265 16449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:29.895256 16449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:29.896549 16449 hybrid_clock.cc:648] HybridClock initialized: now 1786372949896489 us; error 58 us; skew 500 ppm
I20260810 14:42:29.896960 16449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:29 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372949, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:29.900705 16449 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:42:29.902345 16449 webserver.cc:460] Webserver started at http://127.21.55.126:34265/ using document root <none> and password file <none>
I20260810 14:42:29.903162 16449 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:29.903265 16449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:29.909310 16449 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:42:29.914273 16471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:29.916349 16449 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260810 14:42:29.916529 16449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "d0bf2c0428fb4615800a8e628b37861d"
format_stamp: "Formatted at 2026-08-10 14:41:42 on dist-test-slave-h6n0"
server_key: "bf162931067409522fbb674d551c9efd"
server_key_iv: "936bc9fe80a91f7bb36fe495287257b6"
server_key_version: "encryptionkey@0"
I20260810 14:42:29.917527 16449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:29.943146 16449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:29.947697 16449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:29.947959 16449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:29.970268 16449 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:43661
I20260810 14:42:29.970314 16562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:43661 every 8 connection(s)
I20260810 14:42:29.971982 16449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:42:29.972752 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 16449
I20260810 14:42:29.973891 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:46259
--local_ip_for_outbound_sockets=127.21.55.65
--tserver_master_addrs=127.21.55.126:43661
--webserver_port=37441
--webserver_interface=127.21.55.65
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:42:29.979758 16563 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:42:29.983292 16563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Bootstrap starting.
I20260810 14:42:30.005549 16563 log.cc:824] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:30.023499 16563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:42:30.024080 16563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Bootstrap complete.
I20260810 14:42:30.033299 16563 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:42:30.034220 16563 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0bf2c0428fb4615800a8e628b37861d, State: Initialized, Role: FOLLOWER
I20260810 14:42:30.034862 16563 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:42:30.035113 16563 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:42:30.035239 16563 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:42:30.035427 16563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 2 FOLLOWER]: Advancing to term 3
I20260810 14:42:30.039284 16563 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:42:30.039759 16563 leader_election.cc:302] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0bf2c0428fb4615800a8e628b37861d; no voters:
I20260810 14:42:30.040540 16563 leader_election.cc:288] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 14:42:30.040640 16567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 3 FOLLOWER]: Leader election won for term 3
I20260810 14:42:30.041810 16567 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [term 3 LEADER]: Becoming Leader. State: Replica: d0bf2c0428fb4615800a8e628b37861d, State: Running, Role: LEADER
I20260810 14:42:30.042358 16567 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } }
I20260810 14:42:30.042796 16563 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:42:30.048503 16571 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d0bf2c0428fb4615800a8e628b37861d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } } }
I20260810 14:42:30.051532 16571 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: This master's current role is: LEADER
I20260810 14:42:30.047868 16572 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0bf2c0428fb4615800a8e628b37861d. Latest consensus state: current_term: 3 leader_uuid: "d0bf2c0428fb4615800a8e628b37861d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0bf2c0428fb4615800a8e628b37861d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 43661 } } }
I20260810 14:42:30.052956 16572 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d [sys.catalog]: This master's current role is: LEADER
I20260810 14:42:30.059085 16581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:42:30.068213 16581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:42:30.070281 16581 catalog_manager.cc:1259] Loaded cluster ID: 3fcdc3251e3f46a1b7ee2c0a60c5b4e5
I20260810 14:42:30.071137 16581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:42:30.077226 16581 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:42:30.079439 16581 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Loaded TSK: 0
I20260810 14:42:30.079619 16581 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d0bf2c0428fb4615800a8e628b37861d: Loaded TSK: 1
F20260810 14:42:30.081071 16581 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa68c4b39ad google::LogMessage::Fail() at ??:0
@ 0x7fa68c4b688a google::LogMessage::SendToLog() at ??:0
@ 0x7fa68c4b34de google::LogMessage::Flush() at ??:0
@ 0x7fa68c4b4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa6902177d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa69020673e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa6a1ee21c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa6a1ee1783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa6a1f6d2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fa6a1f6d0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fa6a1fb75e9 std::function<>::operator()() at ??:0
I20260810 14:42:30.155673 16606 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fa6a1ee5cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fa6a1ee3c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa6a1f6b610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fa6a1f6b439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa68f0ff345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa68f10291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fa68f102799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa68f0ddeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa6923b76db start_thread at ??:0
@ 0x7fa68b34d61f clone at ??:0
W20260810 14:42:30.452153 16565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:30.452464 16565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:30.452610 16565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:30.460178 16565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:30.460302 16565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:30.460510 16565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:42:30.471037 16565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:46259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=37441
--tserver_master_addrs=127.21.55.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:30.472513 16565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:30.473948 16565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:30.488560 16641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:30.488905 16642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:30.489130 16646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:30.489862 16565 server_base.cc:1034] running on GCE node
I20260810 14:42:30.490638 16565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:30.491724 16565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:30.493005 16565 hybrid_clock.cc:648] HybridClock initialized: now 1786372950492960 us; error 58 us; skew 500 ppm
I20260810 14:42:30.493371 16565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:30 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372950, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:30.498066 16565 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:42:30.499778 16565 webserver.cc:460] Webserver started at http://127.21.55.65:37441/ using document root <none> and password file <none>
I20260810 14:42:30.500510 16565 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:30.500623 16565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:30.507010 16565 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260810 14:42:30.511436 16656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:30.513845 16565 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260810 14:42:30.514171 16565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "580e0549c0b241dd8865e230d10963a9"
format_stamp: "Formatted at 2026-08-10 14:41:43 on dist-test-slave-h6n0"
server_key: "5c107c5a5ddd9780191ffc9d6a33adef"
server_key_iv: "5771bec2725c739cc1e4e193b49968f2"
server_key_version: "encryptionkey@0"
I20260810 14:42:30.515748 16565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:30.531822 16565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:30.536587 16565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:30.536921 16565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:30.538174 16565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:30.540400 16664 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 14:42:30.547698 16565 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 14:42:30.547874 16565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20260810 14:42:30.548025 16565 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 14:42:30.552024 16565 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 14:42:30.552219 16565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260810 14:42:30.552759 16664 tablet_bootstrap.cc:492] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Bootstrap starting.
I20260810 14:42:30.561482 16664 log.cc:824] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:30.579906 16664 tablet_bootstrap.cc:492] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:42:30.580881 16664 tablet_bootstrap.cc:492] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Bootstrap complete.
I20260810 14:42:30.581499 16664 ts_tablet_manager.cc:1397] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Time spent bootstrapping tablet: real 0.029s user 0.018s sys 0.010s
I20260810 14:42:30.592417 16664 raft_consensus.cc:348] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:30.593984 16664 raft_consensus.cc:729] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580e0549c0b241dd8865e230d10963a9, State: Initialized, Role: FOLLOWER
I20260810 14:42:30.594906 16664 consensus_queue.cc:260] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:30.596028 16565 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:46259
I20260810 14:42:30.596095 16813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:46259 every 8 connection(s)
I20260810 14:42:30.596972 16664 ts_tablet_manager.cc:1428] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260810 14:42:30.597301 16565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:42:30.600286 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 16565
I20260810 14:42:30.601188 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:33019
--local_ip_for_outbound_sockets=127.21.55.66
--tserver_master_addrs=127.21.55.126:43661
--webserver_port=37665
--webserver_interface=127.21.55.66
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:42:30.602254 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:30.599975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:54819)
0810 14:42:30.600861 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:30.601161 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":395,"threads_started":1}
W20260810 14:42:30.602988 16817 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:43661 (0 consecutive failures): Network error: Failed to ping master at 127.21.55.126:43661: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
I20260810 14:42:30.603915 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:30.603559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:44959)
0810 14:42:30.603727 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:30.603811 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 14:42:30.605644 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:30.604959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:40257)
0810 14:42:30.605249 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:30.605386 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260810 14:42:30.606029 16817 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:42:30.948173 16821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:30.948472 16821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:30.948623 16821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:30.955910 16821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:30.956018 16821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:30.956210 16821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:42:30.965864 16821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:33019
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=37665
--tserver_master_addrs=127.21.55.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:30.967156 16821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:30.968587 16821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:30.982640 16827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:30.982681 16830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:30.982715 16836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:30.984019 16821 server_base.cc:1034] running on GCE node
I20260810 14:42:30.984732 16821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:30.985733 16821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:30.987005 16821 hybrid_clock.cc:648] HybridClock initialized: now 1786372950986950 us; error 95 us; skew 500 ppm
I20260810 14:42:30.987349 16821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:30 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372950, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:30.990871 16821 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:42:30.992861 16821 webserver.cc:460] Webserver started at http://127.21.55.66:37665/ using document root <none> and password file <none>
I20260810 14:42:30.993611 16821 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:30.993772 16821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:30.999576 16821 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:42:31.002764 16847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:31.004280 16821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:42:31.004487 16821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "2a60da52205e49b09d90da0b303ce671"
format_stamp: "Formatted at 2026-08-10 14:41:43 on dist-test-slave-h6n0"
server_key: "9ba922d800e835b89376f3b9fc8feb4e"
server_key_iv: "2c2fe987edbf123f089aa0d33f918113"
server_key_version: "encryptionkey@0"
I20260810 14:42:31.005523 16821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:31.030850 16821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:31.035503 16821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:31.035799 16821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:31.036875 16821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:31.038268 16821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:31.038414 16821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:31.038575 16821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:31.038684 16821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260810 14:42:31.080880 16821 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:33019
I20260810 14:42:31.080937 17008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:33019 every 8 connection(s)
I20260810 14:42:31.082342 16821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:42:31.084493 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 16821
I20260810 14:42:31.085608 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:36769
--local_ip_for_outbound_sockets=127.21.55.67
--tserver_master_addrs=127.21.55.126:43661
--webserver_port=35107
--webserver_interface=127.21.55.67
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:42:31.090740 17012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:31.087470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:50101)
0810 14:42:31.088614 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:31.088996 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":517,"threads_started":1}
W20260810 14:42:31.091835 17010 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:43661 (0 consecutive failures): Network error: Failed to ping master at 127.21.55.126:43661: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
I20260810 14:42:31.093293 17012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:31.092749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:43855)
0810 14:42:31.092992 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:31.093106 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 14:42:31.094579 17012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:31.094055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:42729)
0810 14:42:31.094240 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:31.094354 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 14:42:31.094907 17010 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:42:31.466399 17013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:31.466715 17013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:31.466814 17013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:31.473940 17013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:31.474052 17013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:31.474174 17013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:42:31.483862 17013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:36769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=35107
--tserver_master_addrs=127.21.55.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:31.485276 17013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:31.486714 17013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:31.502127 17021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:31.502813 17020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:31.503049 17026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:31.503997 17013 server_base.cc:1034] running on GCE node
I20260810 14:42:31.504734 17013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:31.506073 17013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:31.507532 17013 hybrid_clock.cc:648] HybridClock initialized: now 1786372951507420 us; error 105 us; skew 500 ppm
I20260810 14:42:31.507900 17013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:31 dist-test-slave-h6n0 krb5kdc[14791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372951, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:31.511914 17013 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:42:31.513535 17013 webserver.cc:460] Webserver started at http://127.21.55.67:35107/ using document root <none> and password file <none>
I20260810 14:42:31.514247 17013 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:31.514405 17013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:31.521436 17013 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:42:31.524995 17034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:31.527532 17013 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260810 14:42:31.527736 17013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "9fbff42dc4b243f39747ee9030672130"
format_stamp: "Formatted at 2026-08-10 14:41:44 on dist-test-slave-h6n0"
server_key: "1aa03bacc9924dc835558d331b4db2d0"
server_key_iv: "fe7d746c6231f4ef3ada432848ddbecb"
server_key_version: "encryptionkey@0"
I20260810 14:42:31.528823 17013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:31.581362 17013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:31.586165 17013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:31.586464 17013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:31.588114 17013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:31.589596 17013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:31.589736 17013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:31.589880 17013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:31.589983 17013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:31.610181 17094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:31.606592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:41273)
0810 14:42:31.609690 (+ 3098us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:31.609828 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2816,"thread_start_us":2501,"threads_started":1}
I20260810 14:42:31.632161 17013 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:36769
I20260810 14:42:31.632252 17194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:36769 every 8 connection(s)
I20260810 14:42:31.633828 17013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:42:31.636422 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 17013
I20260810 14:42:31.641301 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:31.637755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:38861)
0810 14:42:31.639248 (+ 1493us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:31.639856 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":922,"thread_start_us":554,"threads_started":1}
W20260810 14:42:31.642388 17195 heartbeater.cc:644] Failed to heartbeat to 127.21.55.126:43661 (0 consecutive failures): Network error: Failed to ping master at 127.21.55.126:43661: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
I20260810 14:42:31.643448 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:31.643099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:39969)
0810 14:42:31.643273 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:31.643343 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 14:42:31.644304 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:31.643966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:43247)
0810 14:42:31.644119 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:31.644194 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:42:31.644556 17195 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:42:31.782339 17203 raft_consensus.cc:482] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:42:31.782879 17203 raft_consensus.cc:504] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:31.784782 17203 leader_election.cc:288] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769), 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019)
Aug 10 14:42:31 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372950, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
Aug 10 14:42:31 dist-test-slave-h6n0 krb5kdc[14791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372950, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:42:31.816560 17094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:31.785146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:36769 (local address 127.21.55.65:42255)
0810 14:42:31.785397 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:31.785437 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:31.786052 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:31.790671 (+ 4619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:31.790683 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:31.791555 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:31.792378 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:31.792400 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:31.794011 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:31.794015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:31.795768 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:31.795795 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:31.797360 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:31.797369 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:31.797843 (+ 474us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:31.804467 (+ 6624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:42:31.804485 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:31.807642 (+ 3157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:31.811782 (+ 4140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:31.811791 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:31.811807 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:31.812628 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:31.813663 (+ 1035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:31.813668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:31.813673 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:31.813985 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:31.815037 (+ 1052us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:31.815042 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:31.815267 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 14:42:31.815527 (+ 260us) client_negotiation.cc:240] Negotiation successful
0810 14:42:31.816097 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":78,"spinlock_wait_cycles":28544}
I20260810 14:42:31.816592 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:31.785568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:42255 (local address 127.21.55.67:36769)
0810 14:42:31.787666 (+ 2098us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:31.787678 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:31.787716 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:42:31.789866 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:31.789876 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:31.790258 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:31.790601 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:31.792602 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:31.793806 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:31.796064 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:31.797031 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:31.797285 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:31.807938 (+ 10653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:31.807953 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:31.807965 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:31.808006 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:31.811610 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:31.812968 (+ 1358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:31.812983 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:31.812994 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:31.813519 (+ 525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:31.814128 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:31.814152 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:31.814159 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:31.814882 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:31.815112 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:31.815778 (+ 666us) server_negotiation.cc:299] Negotiation successful
0810 14:42:31.816176 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1925,"thread_start_us":1779,"threads_started":1}
I20260810 14:42:31.816606 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:31.785985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:44605 (local address 127.21.55.66:33019)
0810 14:42:31.787260 (+ 1275us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:31.787278 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:31.787325 (+ 47us) server_negotiation.cc:407] Connection header received
0810 14:42:31.789574 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:31.789588 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:31.790067 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:31.790454 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:31.792602 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:31.793865 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:31.796098 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:31.797065 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:31.797492 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:31.807701 (+ 10209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:31.807710 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:31.807723 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:31.807766 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:31.811610 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:31.812968 (+ 1358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:31.812983 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:31.812994 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:31.813512 (+ 518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:31.814153 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:31.814158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:31.814162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:31.814799 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:31.815016 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:31.815782 (+ 766us) server_negotiation.cc:299] Negotiation successful
0810 14:42:31.816168 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1093,"thread_start_us":978,"threads_started":1}
I20260810 14:42:31.816995 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:31.785632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:33019 (local address 127.21.55.65:44605)
0810 14:42:31.787036 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:31.787059 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:31.787191 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:31.790671 (+ 3480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:31.790683 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:31.791672 (+ 989us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:31.792379 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:31.792400 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:31.794188 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:31.794204 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:31.795842 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:31.795861 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:31.797537 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:31.797546 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:31.797851 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:31.804428 (+ 6577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:42:31.804462 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:31.807365 (+ 2903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:31.811782 (+ 4417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:31.811791 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:31.811807 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:31.812628 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:31.813709 (+ 1081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:31.813713 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:31.813715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:31.813993 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:31.814953 (+ 960us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:31.814961 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:31.815259 (+ 298us) client_negotiation.cc:769] Sending connection context
0810 14:42:31.815527 (+ 268us) client_negotiation.cc:240] Negotiation successful
0810 14:42:31.816269 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1240,"mutex_wait_us":261,"spinlock_wait_cycles":36352,"thread_start_us":1039,"threads_started":1}
W20260810 14:42:31.820036 16660 leader_election.cc:341] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769): Not found: Tablet not found: 2aa509261fe74a758decf59406b11ca5
W20260810 14:42:31.820500 16660 leader_election.cc:341] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019): Not found: Tablet not found: 2aa509261fe74a758decf59406b11ca5
I20260810 14:42:31.820624 16660 leader_election.cc:302] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 580e0549c0b241dd8865e230d10963a9; no voters: 2a60da52205e49b09d90da0b303ce671, 9fbff42dc4b243f39747ee9030672130
I20260810 14:42:31.820977 17203 raft_consensus.cc:2726] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:42:32.096908 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:32.095675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:48251)
0810 14:42:32.096383 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:32.096608 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":397,"threads_started":1}
I20260810 14:42:32.612331 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:32.611361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:53149)
0810 14:42:32.611962 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:32.612077 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":369,"threads_started":1}
I20260810 14:42:32.645955 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:32.645123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:48201)
0810 14:42:32.645550 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:32.645704 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":213,"threads_started":1}
I20260810 14:42:33.098937 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:33.098186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:60977)
0810 14:42:33.098687 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:33.098776 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":246,"threads_started":1}
I20260810 14:42:33.614274 17212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:33.613416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:42995)
0810 14:42:33.613991 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:33.614080 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":333,"threads_started":1}
I20260810 14:42:33.647818 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:33.647016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:49065)
0810 14:42:33.647538 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:33.647629 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":269,"threads_started":1}
I20260810 14:42:33.747136 17214 raft_consensus.cc:482] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:42:33.747504 17214 raft_consensus.cc:504] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:33.748369 17214 leader_election.cc:288] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769), 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019)
W20260810 14:42:33.749533 16660 leader_election.cc:341] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769): Not found: Tablet not found: 2aa509261fe74a758decf59406b11ca5
W20260810 14:42:33.749720 16660 leader_election.cc:341] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019): Not found: Tablet not found: 2aa509261fe74a758decf59406b11ca5
I20260810 14:42:33.749794 16660 leader_election.cc:302] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 580e0549c0b241dd8865e230d10963a9; no voters: 2a60da52205e49b09d90da0b303ce671, 9fbff42dc4b243f39747ee9030672130
I20260810 14:42:33.750022 17214 raft_consensus.cc:2726] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:42:34.101204 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:34.100030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:42671)
0810 14:42:34.100744 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:34.100918 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":355,"threads_started":1}
I20260810 14:42:34.616292 17218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:34.615248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:45069)
0810 14:42:34.615933 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:34.616071 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":363,"threads_started":1}
I20260810 14:42:34.650019 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:34.649077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:50999)
0810 14:42:34.649712 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:34.649827 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":313,"threads_started":1}
I20260810 14:42:35.103260 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:35.102299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:43723)
0810 14:42:35.102959 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:35.103056 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":401,"threads_started":1}
I20260810 14:42:35.618224 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:35.617373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:47267)
0810 14:42:35.617910 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:35.618020 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":270,"threads_started":1}
I20260810 14:42:35.651793 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:35.651032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:34447)
0810 14:42:35.651542 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:35.651623 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":250,"threads_started":1}
I20260810 14:42:36.105239 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:36.104444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:49313)
0810 14:42:36.104952 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:36.105067 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":267,"threads_started":1}
I20260810 14:42:36.411345 17228 raft_consensus.cc:482] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:42:36.411650 17228 raft_consensus.cc:504] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:36.412423 17228 leader_election.cc:288] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769), 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019)
W20260810 14:42:36.413745 16660 leader_election.cc:341] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019): Not found: Tablet not found: 2aa509261fe74a758decf59406b11ca5
W20260810 14:42:36.414182 16660 leader_election.cc:341] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769): Not found: Tablet not found: 2aa509261fe74a758decf59406b11ca5
I20260810 14:42:36.414309 16660 leader_election.cc:302] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 580e0549c0b241dd8865e230d10963a9; no voters: 2a60da52205e49b09d90da0b303ce671, 9fbff42dc4b243f39747ee9030672130
I20260810 14:42:36.414606 17228 raft_consensus.cc:2726] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:42:36.620435 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:36.619155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:47383)
0810 14:42:36.619970 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:36.620180 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":380,"threads_started":1}
I20260810 14:42:36.653949 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:36.652898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:35405)
0810 14:42:36.653594 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:36.653729 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":388,"threads_started":1}
I20260810 14:42:37.107089 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:37.106242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:59147)
0810 14:42:37.106803 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:37.106890 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":321,"threads_started":1}
I20260810 14:42:37.622900 17236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:37.621869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:32909)
0810 14:42:37.622447 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:37.622600 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":319,"threads_started":1}
I20260810 14:42:37.656035 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:37.655222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:59429)
0810 14:42:37.655747 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:37.655839 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":269,"threads_started":1}
I20260810 14:42:38.109274 17239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:38.108319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:60693)
0810 14:42:38.108933 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:38.109032 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":304,"threads_started":1}
I20260810 14:42:38.365391 17240 raft_consensus.cc:482] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:42:38.365655 17240 raft_consensus.cc:504] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:38.366396 17240 leader_election.cc:288] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769), 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019)
W20260810 14:42:38.367813 16660 leader_election.cc:341] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019): Not found: Tablet not found: 2aa509261fe74a758decf59406b11ca5
W20260810 14:42:38.368074 16660 leader_election.cc:341] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769): Not found: Tablet not found: 2aa509261fe74a758decf59406b11ca5
I20260810 14:42:38.368163 16660 leader_election.cc:302] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 580e0549c0b241dd8865e230d10963a9; no voters: 2a60da52205e49b09d90da0b303ce671, 9fbff42dc4b243f39747ee9030672130
I20260810 14:42:38.368482 17240 raft_consensus.cc:2726] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:42:38.625191 17241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:38.623924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:32851)
0810 14:42:38.624728 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:38.624911 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":437,"threads_started":1}
I20260810 14:42:38.658401 17242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:38.657380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:38927)
0810 14:42:38.658077 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:38.658172 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":304,"threads_started":1}
I20260810 14:42:39.111658 17244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:39.110643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:54491)
0810 14:42:39.111295 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:39.111411 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":300,"threads_started":1}
I20260810 14:42:39.627750 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:39.626470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:53599)
0810 14:42:39.627287 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:39.627417 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":441,"threads_started":1}
I20260810 14:42:39.660645 17248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:39.659770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:52181)
0810 14:42:39.660358 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:39.660451 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":344,"threads_started":1}
I20260810 14:42:40.113953 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:40.112860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:43859)
0810 14:42:40.113547 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:40.113740 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":377,"threads_started":1}
I20260810 14:42:40.629845 17251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:40.629060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:59097)
0810 14:42:40.629562 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:40.629658 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":259,"threads_started":1}
I20260810 14:42:40.662906 17260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:40.662021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:53345)
0810 14:42:40.662603 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:40.662716 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":251,"threads_started":1}
I20260810 14:42:41.115928 17261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:41.115039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:38663)
0810 14:42:41.115619 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:41.115715 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":315,"threads_started":1}
I20260810 14:42:41.632313 17262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:41.631468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:54821)
0810 14:42:41.632028 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:41.632128 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":301,"threads_started":1}
I20260810 14:42:41.664908 17264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:41.663903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:46971)
0810 14:42:41.664633 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:41.664730 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":532,"threads_started":1}
I20260810 14:42:42.117671 17265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:42.116881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:35951)
0810 14:42:42.117404 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:42.117502 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":284,"threads_started":1}
I20260810 14:42:42.634327 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:42.633377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:40397)
0810 14:42:42.634096 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:42.634176 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":378,"threads_started":1}
I20260810 14:42:42.666889 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:42.665950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:36113)
0810 14:42:42.666587 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:42.666695 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":298,"threads_started":1}
I20260810 14:42:43.119450 17270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:43.118680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:38297)
0810 14:42:43.119189 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:43.119272 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":278,"threads_started":1}
I20260810 14:42:43.636387 17274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:43.635359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:51857)
0810 14:42:43.636070 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:43.636190 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":392,"threads_started":1}
I20260810 14:42:43.669668 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:43.668525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:39883)
0810 14:42:43.669265 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:43.669375 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":437,"threads_started":1}
I20260810 14:42:44.122181 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:44.120683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:49847)
0810 14:42:44.121846 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:44.121957 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1061,"thread_start_us":368,"threads_started":1}
I20260810 14:42:44.638768 17279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:44.637617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:42297)
0810 14:42:44.638376 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:44.638482 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":484,"threads_started":1}
I20260810 14:42:44.672147 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:44.671057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:42011)
0810 14:42:44.671800 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:44.671927 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":421,"threads_started":1}
I20260810 14:42:45.124233 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:45.123277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:34315)
0810 14:42:45.123863 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:45.123978 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":313,"threads_started":1}
I20260810 14:42:45.597604 17282 raft_consensus.cc:482] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:42:45.597903 17282 raft_consensus.cc:504] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fbff42dc4b243f39747ee9030672130" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36769 } } peers { permanent_uuid: "2a60da52205e49b09d90da0b303ce671" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 33019 } } peers { permanent_uuid: "580e0549c0b241dd8865e230d10963a9" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 46259 } }
I20260810 14:42:45.598759 17282 leader_election.cc:288] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769), 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019)
W20260810 14:42:45.599925 16660 leader_election.cc:341] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9fbff42dc4b243f39747ee9030672130 (127.21.55.67:36769): Not found: Tablet not found: 2aa509261fe74a758decf59406b11ca5
W20260810 14:42:45.600164 16660 leader_election.cc:341] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a60da52205e49b09d90da0b303ce671 (127.21.55.66:33019): Not found: Tablet not found: 2aa509261fe74a758decf59406b11ca5
I20260810 14:42:45.600288 16660 leader_election.cc:302] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 580e0549c0b241dd8865e230d10963a9; no voters: 2a60da52205e49b09d90da0b303ce671, 9fbff42dc4b243f39747ee9030672130
I20260810 14:42:45.600565 17282 raft_consensus.cc:2726] T 2aa509261fe74a758decf59406b11ca5 P 580e0549c0b241dd8865e230d10963a9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:42:45.640995 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:45.639961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:38765)
0810 14:42:45.640606 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:45.640742 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":376,"threads_started":1}
I20260810 14:42:45.674046 17285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:45.673260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:60501)
0810 14:42:45.673741 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:45.673829 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":257,"threads_started":1}
I20260810 14:42:46.126775 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:46.125845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.66:60655)
0810 14:42:46.126412 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:46.126586 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":332,"threads_started":1}
I20260810 14:42:46.642068 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 16565
I20260810 14:42:46.643949 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:46.642419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.65:50297)
0810 14:42:46.643559 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:46.643682 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":968,"thread_start_us":824,"threads_started":1}
I20260810 14:42:46.660655 17290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:46.643829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52580 (local address 127.21.55.65:46259)
0810 14:42:46.644601 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:46.644611 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:46.644648 (+ 37us) server_negotiation.cc:407] Connection header received
0810 14:42:46.644721 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:46.644728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:46.644978 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:46.645129 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:46.647048 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:46.648277 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:46.649954 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:46.650817 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:46.650954 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:46.653726 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:46.653741 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:46.653755 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:46.653800 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:46.656903 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:46.657893 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:46.657902 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:46.657904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:46.658365 (+ 461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:46.658928 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:46.658931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:46.658934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:46.659348 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:46.659613 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:46.659983 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 14:42:46.660230 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":465,"threads_started":1}
I20260810 14:42:46.662066 16808 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:46.676458 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:46.675519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:43661 (local address 127.21.55.67:40275)
0810 14:42:46.676087 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:46.676219 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":268,"threads_started":1}
I20260810 14:42:46.757392 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 16565
I20260810 14:42:46.783892 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 16821
I20260810 14:42:46.800508 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:46.785222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56102 (local address 127.21.55.66:33019)
0810 14:42:46.785737 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:46.785741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:46.785757 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:42:46.785824 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:46.785827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:46.785884 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:46.786019 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:46.787484 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:46.788286 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:46.789446 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:46.789887 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:46.790030 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:46.793909 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:46.793916 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:46.793920 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:46.793976 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:46.797576 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:46.798367 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:46.798373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:46.798377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:46.798593 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:46.798974 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:46.798979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:46.798983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:46.799276 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:46.799580 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:46.799871 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 14:42:46.800123 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":263,"threads_started":1}
I20260810 14:42:46.801548 17002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:46.893252 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 16821
I20260810 14:42:46.928709 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 17013
I20260810 14:42:46.946700 17297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:46.929625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.21.55.67:36769)
0810 14:42:46.930673 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:46.930680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:46.930708 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:42:46.930788 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:46.930793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:46.930883 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:46.931026 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:46.933383 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:46.934851 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:46.936951 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:46.937481 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:46.937592 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:46.941223 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:46.941229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:46.941232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:46.941264 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:46.943914 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:46.944473 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:46.944477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:46.944479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:46.944624 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:46.945322 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:46.945325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:46.945327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:46.945593 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:46.945812 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:46.946164 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 14:42:46.946329 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":466,"threads_started":1}
I20260810 14:42:46.947643 17186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:47.035630 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 17013
2026-08-10T14:42:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (66799 ms)
[----------] 30 tests from SecurityITest (325905 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:42:47 dist-test-slave-h6n0 krb5kdc[17302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:42:47 dist-test-slave-h6n0 krb5kdc[17302](info): set up 2 sockets
Aug 10 14:42:47 dist-test-slave-h6n0 krb5kdc[17302](info): commencing operation
krb5kdc: starting...
W20260810 14:42:48.477990 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.386s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:42:48 dist-test-slave-h6n0 krb5kdc[17302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:42:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:42:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:42:48.805467 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38641
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:41149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:38641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:42:49.163415 17320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:49.163786 17320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:49.163968 17320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:49.173128 17320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:42:49.173300 17320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:49.173417 17320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:49.173527 17320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:42:49.173594 17320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:42:49.183570 17320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:38641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:49.185238 17320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:49.186662 17320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:49.199775 17326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:49.199829 17328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:49.199864 17334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:49.200232 17320 server_base.cc:1034] running on GCE node
I20260810 14:42:49.201249 17320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:49.202769 17320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:49.204001 17320 hybrid_clock.cc:648] HybridClock initialized: now 1786372969203935 us; error 39 us; skew 500 ppm
I20260810 14:42:49.204371 17320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:49 dist-test-slave-h6n0 krb5kdc[17302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372969, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:49.213243 17320 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:42:49.215153 17320 webserver.cc:460] Webserver started at http://127.21.55.126:45105/ using document root <none> and password file <none>
I20260810 14:42:49.215922 17320 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:49.216063 17320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:49.216409 17320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:49.226867 17320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "27c88dc7550f429686c50063ee111529"
format_stamp: "Formatted at 2026-08-10 14:42:49 on dist-test-slave-h6n0"
server_key: "dee01c4ea0cc39eee5dcff398c2dc123"
server_key_iv: "89408c3fdb0ec101943594b09331cb61"
server_key_version: "encryptionkey@0"
I20260810 14:42:49.228083 17320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "27c88dc7550f429686c50063ee111529"
format_stamp: "Formatted at 2026-08-10 14:42:49 on dist-test-slave-h6n0"
server_key: "dee01c4ea0cc39eee5dcff398c2dc123"
server_key_iv: "89408c3fdb0ec101943594b09331cb61"
server_key_version: "encryptionkey@0"
I20260810 14:42:49.234965 17320 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260810 14:42:49.240345 17351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:49.242141 17320 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:42:49.242340 17320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "27c88dc7550f429686c50063ee111529"
format_stamp: "Formatted at 2026-08-10 14:42:49 on dist-test-slave-h6n0"
server_key: "dee01c4ea0cc39eee5dcff398c2dc123"
server_key_iv: "89408c3fdb0ec101943594b09331cb61"
server_key_version: "encryptionkey@0"
I20260810 14:42:49.242689 17320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:49.260902 17320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:49.266599 17320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:49.266932 17320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:49.289548 17320 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:38641
I20260810 14:42:49.289595 17432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:38641 every 8 connection(s)
I20260810 14:42:49.291097 17320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:42:49.295898 17433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:49.299175 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 17320
I20260810 14:42:49.299460 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:42:49.300220 21725 external_mini_cluster.cc:1428] Setting key f4ca36648ae613c4cff6d513a607eb09
I20260810 14:42:49.310518 17433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529: Bootstrap starting.
I20260810 14:42:49.314522 17433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:49.315580 17433 log.cc:824] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:42:49 dist-test-slave-h6n0 krb5kdc[17302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:42:49.318655 17433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529: No bootstrap required, opened a new log
I20260810 14:42:49.327412 17439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:49.302721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33706 (local address 127.21.55.126:38641)
0810 14:42:49.303616 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:49.303631 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:49.303687 (+ 56us) server_negotiation.cc:407] Connection header received
0810 14:42:49.305790 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:49.305828 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:49.306378 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:49.306882 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:49.308349 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:49.310461 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:49.311896 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:49.313031 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:49.313208 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:49.318155 (+ 4947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:49.318168 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:49.318186 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:49.318234 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:49.322339 (+ 4105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:49.323154 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:49.323162 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:49.323170 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:49.323393 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:49.323793 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:49.323799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:49.323802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:49.324655 (+ 853us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:49.324982 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:49.325508 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 14:42:49.325988 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":379,"threads_started":1}
I20260810 14:42:49.329869 17433 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c88dc7550f429686c50063ee111529" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38641 } }
I20260810 14:42:49.330268 17433 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:49.330399 17433 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c88dc7550f429686c50063ee111529, State: Initialized, Role: FOLLOWER
I20260810 14:42:49.331099 17433 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c88dc7550f429686c50063ee111529" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38641 } }
I20260810 14:42:49.331367 17433 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:42:49.331507 17433 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:42:49.331684 17433 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:49.335508 17433 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c88dc7550f429686c50063ee111529" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38641 } }
I20260810 14:42:49.336364 17433 leader_election.cc:302] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27c88dc7550f429686c50063ee111529; no voters:
I20260810 14:42:49.337199 17433 leader_election.cc:288] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:42:49.337430 17441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:42:49.338400 17441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [term 1 LEADER]: Becoming Leader. State: Replica: 27c88dc7550f429686c50063ee111529, State: Running, Role: LEADER
I20260810 14:42:49.339237 17441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c88dc7550f429686c50063ee111529" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38641 } }
I20260810 14:42:49.340288 17433 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:42:49.342651 17443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27c88dc7550f429686c50063ee111529" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c88dc7550f429686c50063ee111529" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38641 } } }
I20260810 14:42:49.343092 17444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27c88dc7550f429686c50063ee111529. Latest consensus state: current_term: 1 leader_uuid: "27c88dc7550f429686c50063ee111529" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27c88dc7550f429686c50063ee111529" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38641 } } }
I20260810 14:42:49.343485 17443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [sys.catalog]: This master's current role is: LEADER
I20260810 14:42:49.344833 17444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529 [sys.catalog]: This master's current role is: LEADER
I20260810 14:42:49.347332 17455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:42:49.352919 17455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:42:49.363741 17455 catalog_manager.cc:1347] Generated new cluster ID: 644e583bec4a436daa31c140b12953cd
I20260810 14:42:49.363897 17455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:42:49.382416 17455 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:42:49.384289 17455 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:42:49.392815 17455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529: Generated new TSK 0
I20260810 14:42:49.394050 17455 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:42:49.482681 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38641
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:41149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:42:49.869715 17473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:49.870041 17473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:49.870222 17473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:49.880138 17473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:49.880264 17473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:49.880446 17473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:42:49.890820 17473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:49.892346 17473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:49.893720 17473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:49.911648 17483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:49.911602 17480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:49.911692 17479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:49.912945 17473 server_base.cc:1034] running on GCE node
I20260810 14:42:49.913692 17473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:49.914811 17473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:49.916215 17473 hybrid_clock.cc:648] HybridClock initialized: now 1786372969916159 us; error 30 us; skew 500 ppm
I20260810 14:42:49.916617 17473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:49 dist-test-slave-h6n0 krb5kdc[17302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372969, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:49.920955 17473 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:42:49.923213 17473 webserver.cc:460] Webserver started at http://127.21.55.65:33859/ using document root <none> and password file <none>
I20260810 14:42:49.924214 17473 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:49.924371 17473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:49.924752 17473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:49.929459 17473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "22049959f11d4ebeacfb795bf2862c55"
format_stamp: "Formatted at 2026-08-10 14:42:49 on dist-test-slave-h6n0"
server_key: "cf2cbc8efcff56d00cf485f1199b35bd"
server_key_iv: "097d349ad46fdec9ed099e35e8de8492"
server_key_version: "encryptionkey@0"
I20260810 14:42:49.930872 17473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "22049959f11d4ebeacfb795bf2862c55"
format_stamp: "Formatted at 2026-08-10 14:42:49 on dist-test-slave-h6n0"
server_key: "cf2cbc8efcff56d00cf485f1199b35bd"
server_key_iv: "097d349ad46fdec9ed099e35e8de8492"
server_key_version: "encryptionkey@0"
I20260810 14:42:49.938704 17473 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260810 14:42:49.943416 17495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:49.945604 17473 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260810 14:42:49.945861 17473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "22049959f11d4ebeacfb795bf2862c55"
format_stamp: "Formatted at 2026-08-10 14:42:49 on dist-test-slave-h6n0"
server_key: "cf2cbc8efcff56d00cf485f1199b35bd"
server_key_iv: "097d349ad46fdec9ed099e35e8de8492"
server_key_version: "encryptionkey@0"
I20260810 14:42:49.946197 17473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:49.969437 17473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:49.973874 17473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:49.974191 17473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:49.975360 17473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:49.976994 17473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:49.977138 17473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 14:42:49.977288 17473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:49.977363 17473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:50.020707 17473 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:44485
I20260810 14:42:50.020766 17665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:44485 every 8 connection(s)
I20260810 14:42:50.022758 17473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:42:50.031627 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 17473
I20260810 14:42:50.031881 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:42:50.032722 21725 external_mini_cluster.cc:1428] Setting key e50696a4d6d57cfa26deafdb33b11f97
Aug 10 14:42:50 dist-test-slave-h6n0 krb5kdc[17302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372969, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:42:50.050514 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:50.026108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:35735 (local address 127.21.55.126:38641)
0810 14:42:50.026762 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:50.026767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:50.028126 (+ 1359us) server_negotiation.cc:407] Connection header received
0810 14:42:50.030146 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:50.030151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:50.030252 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:50.030378 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:50.032675 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:50.033989 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:50.036346 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:50.036768 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:50.036906 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:50.042337 (+ 5431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:50.042342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:50.042346 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:50.042382 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:50.046653 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:50.047630 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:50.047636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:50.047640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:50.047958 (+ 318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:50.048569 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:50.048575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:50.048578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:50.048914 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:50.049083 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:50.049818 (+ 735us) server_negotiation.cc:299] Negotiation successful
0810 14:42:50.050049 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":305,"threads_started":1}
I20260810 14:42:50.051600 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:50.026887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38641 (local address 127.21.55.65:35735)
0810 14:42:50.027806 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:50.027851 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:50.029839 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:50.030653 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:50.030662 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:50.031427 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:50.032316 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:50.032351 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:50.034209 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:50.034214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:50.036158 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:50.036177 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:50.037041 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:50.037051 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:50.037406 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:50.038395 (+ 989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:42:50.038430 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:50.042065 (+ 3635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:50.046797 (+ 4732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:50.046808 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:50.046845 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:50.047495 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:50.048106 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:50.048111 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:50.048115 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:50.048328 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:50.049057 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:50.049066 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:50.049431 (+ 365us) client_negotiation.cc:769] Sending connection context
0810 14:42:50.049711 (+ 280us) client_negotiation.cc:240] Negotiation successful
0810 14:42:50.050271 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":362,"threads_started":1}
I20260810 14:42:50.053175 17667 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38641
I20260810 14:42:50.053515 17667 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:50.054414 17667 heartbeater.cc:511] Master 127.21.55.126:38641 requested a full tablet report, sending...
I20260810 14:42:50.056877 17374 ts_manager.cc:194] Registered new tserver with Master: 22049959f11d4ebeacfb795bf2862c55 (127.21.55.65:44485)
I20260810 14:42:50.058492 17374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:35735
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:42:50.104282 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38641
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:41149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:42:50.472474 17674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:50.472765 17674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:50.472862 17674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:50.480002 17674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:50.480104 17674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:50.480213 17674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:42:50.489753 17674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:50.490971 17674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:50.492172 17674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:50.505942 17683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:50.506088 17687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:50.506048 17685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:50.506762 17674 server_base.cc:1034] running on GCE node
I20260810 14:42:50.507421 17674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:50.508349 17674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:50.509579 17674 hybrid_clock.cc:648] HybridClock initialized: now 1786372970509509 us; error 57 us; skew 500 ppm
I20260810 14:42:50.509867 17674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:50 dist-test-slave-h6n0 krb5kdc[17302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372970, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:50.513404 17674 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:42:50.514983 17674 webserver.cc:460] Webserver started at http://127.21.55.66:46583/ using document root <none> and password file <none>
I20260810 14:42:50.515659 17674 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:50.515770 17674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:50.516060 17674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:50.519299 17674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "31f23268678248bdaffc01a256aca164"
format_stamp: "Formatted at 2026-08-10 14:42:50 on dist-test-slave-h6n0"
server_key: "b4dd2741731977043ca93c23e3151b85"
server_key_iv: "5f39db037c5352954f335233d4900633"
server_key_version: "encryptionkey@0"
I20260810 14:42:50.520063 17674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "31f23268678248bdaffc01a256aca164"
format_stamp: "Formatted at 2026-08-10 14:42:50 on dist-test-slave-h6n0"
server_key: "b4dd2741731977043ca93c23e3151b85"
server_key_iv: "5f39db037c5352954f335233d4900633"
server_key_version: "encryptionkey@0"
I20260810 14:42:50.526386 17674 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 14:42:50.531162 17696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:50.532984 17674 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:42:50.533281 17674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "31f23268678248bdaffc01a256aca164"
format_stamp: "Formatted at 2026-08-10 14:42:50 on dist-test-slave-h6n0"
server_key: "b4dd2741731977043ca93c23e3151b85"
server_key_iv: "5f39db037c5352954f335233d4900633"
server_key_version: "encryptionkey@0"
I20260810 14:42:50.533509 17674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:50.550021 17674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:50.554737 17674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:50.554982 17674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:50.555867 17674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:50.557165 17674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:50.557241 17674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:50.557328 17674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:50.557377 17674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:50.593681 17674 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:38487
I20260810 14:42:50.593783 17876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:38487 every 8 connection(s)
I20260810 14:42:50.595203 17674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:42:50.598501 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 17674
I20260810 14:42:50.598762 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:42:50.599341 21725 external_mini_cluster.cc:1428] Setting key 9ef70d6b59335d2e16831609c93f31af
Aug 10 14:42:50 dist-test-slave-h6n0 krb5kdc[17302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372970, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:42:50.621774 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:50.598303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:59285 (local address 127.21.55.126:38641)
0810 14:42:50.599081 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:50.599091 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:50.600393 (+ 1302us) server_negotiation.cc:407] Connection header received
0810 14:42:50.603090 (+ 2697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:50.603094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:50.603156 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:50.603265 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:50.605385 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:50.606168 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:50.608686 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:50.609281 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:50.609388 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:50.614310 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:50.614314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:50.614318 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:50.614349 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:50.617770 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:50.618874 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:50.618879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:50.618882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:50.619043 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:50.619568 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:50.619572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:50.619574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:50.619896 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:50.620098 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:50.620919 (+ 821us) server_negotiation.cc:299] Negotiation successful
0810 14:42:50.621197 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":445,"threads_started":1}
I20260810 14:42:50.623332 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:50.599254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38641 (local address 127.21.55.66:59285)
0810 14:42:50.600185 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:50.600230 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:50.602790 (+ 2560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:50.603536 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:50.603549 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:50.604389 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:50.605197 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:50.605223 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:50.606340 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:50.606348 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:50.608524 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:50.608544 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:50.609796 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:50.609813 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:50.610201 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:50.611011 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:42:50.611045 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:50.614061 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:50.618023 (+ 3962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:50.618034 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:50.618052 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:50.618699 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:50.619174 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:50.619179 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:50.619183 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:50.619422 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:50.620041 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:50.620051 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:50.620496 (+ 445us) client_negotiation.cc:769] Sending connection context
0810 14:42:50.620811 (+ 315us) client_negotiation.cc:240] Negotiation successful
0810 14:42:50.621669 (+ 858us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":387,"threads_started":1}
I20260810 14:42:50.625033 17878 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38641
I20260810 14:42:50.625406 17878 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:50.626459 17878 heartbeater.cc:511] Master 127.21.55.126:38641 requested a full tablet report, sending...
I20260810 14:42:50.628424 17374 ts_manager.cc:194] Registered new tserver with Master: 31f23268678248bdaffc01a256aca164 (127.21.55.66:38487)
I20260810 14:42:50.629681 17374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:59285
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:42:50.679837 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38641
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:41149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:42:51.050719 17886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:51.051146 17886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:51.051304 17886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:51.059530 17886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:51.059733 17886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:51.059890 17886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:42:51.062441 17667 heartbeater.cc:503] Master 127.21.55.126:38641 was elected leader, sending a full tablet report...
I20260810 14:42:51.072104 17886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:38641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:51.073830 17886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:51.075582 17886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:51.091295 17892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:51.091380 17891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:51.091354 17894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:51.096886 17886 server_base.cc:1034] running on GCE node
I20260810 14:42:51.097628 17886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:51.098819 17886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:51.100131 17886 hybrid_clock.cc:648] HybridClock initialized: now 1786372971100077 us; error 53 us; skew 500 ppm
I20260810 14:42:51.100493 17886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:51 dist-test-slave-h6n0 krb5kdc[17302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372971, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:51.104646 17886 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:42:51.106885 17886 webserver.cc:460] Webserver started at http://127.21.55.67:37575/ using document root <none> and password file <none>
I20260810 14:42:51.107725 17886 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:51.107908 17886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:51.108305 17886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:51.111675 17886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0"
format_stamp: "Formatted at 2026-08-10 14:42:51 on dist-test-slave-h6n0"
server_key: "fba040c2fc8c142cd96845d9a70f9bdc"
server_key_iv: "db1ecf01412c63b926303340cc2cec4d"
server_key_version: "encryptionkey@0"
I20260810 14:42:51.112566 17886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0"
format_stamp: "Formatted at 2026-08-10 14:42:51 on dist-test-slave-h6n0"
server_key: "fba040c2fc8c142cd96845d9a70f9bdc"
server_key_iv: "db1ecf01412c63b926303340cc2cec4d"
server_key_version: "encryptionkey@0"
I20260810 14:42:51.119259 17886 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 14:42:51.124812 17904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:51.126339 17886 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260810 14:42:51.126600 17886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0"
format_stamp: "Formatted at 2026-08-10 14:42:51 on dist-test-slave-h6n0"
server_key: "fba040c2fc8c142cd96845d9a70f9bdc"
server_key_iv: "db1ecf01412c63b926303340cc2cec4d"
server_key_version: "encryptionkey@0"
I20260810 14:42:51.126842 17886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:51.147112 17886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:51.153162 17886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:51.153494 17886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:51.154587 17886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:51.156060 17886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:51.156174 17886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:51.156337 17886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:51.156414 17886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:51.199031 17886 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:43165
I20260810 14:42:51.199229 18052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:43165 every 8 connection(s)
I20260810 14:42:51.200584 17886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:42:51.209592 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 17886
I20260810 14:42:51.209928 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:42:51.210513 21725 external_mini_cluster.cc:1428] Setting key d18a6ae8d6a63e06f3426ff38d25b1f6
Aug 10 14:42:51 dist-test-slave-h6n0 krb5kdc[17302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372971, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:42:51.222013 18055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.203424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:41543 (local address 127.21.55.126:38641)
0810 14:42:51.204171 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:51.204176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:51.204802 (+ 626us) server_negotiation.cc:407] Connection header received
0810 14:42:51.206697 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:51.206702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:51.206807 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:51.206931 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:51.208556 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.210150 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.211988 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.212593 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.212746 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.216324 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:51.216329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:51.216332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:51.216362 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:51.218998 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:51.219710 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:51.219722 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:51.219725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:51.219846 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:51.220284 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:51.220289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:51.220295 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:51.220569 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:51.220744 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:51.221390 (+ 646us) server_negotiation.cc:299] Negotiation successful
0810 14:42:51.221618 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":466,"threads_started":1}
I20260810 14:42:51.222924 18056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.203780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:38641 (local address 127.21.55.67:41543)
0810 14:42:51.204620 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:51.204667 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:51.206386 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:51.207186 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:51.207194 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:51.207735 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:51.208348 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.208368 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.210314 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.210318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.211833 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.211846 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.212841 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.212848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.213100 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.213816 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:42:51.213843 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:51.216096 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:51.219151 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:51.219159 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:51.219173 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:51.219578 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:51.219962 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:51.219967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:51.219971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:51.220158 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:51.220704 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:51.220711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:51.221032 (+ 321us) client_negotiation.cc:769] Sending connection context
0810 14:42:51.221293 (+ 261us) client_negotiation.cc:240] Negotiation successful
0810 14:42:51.221781 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":363,"threads_started":1}
I20260810 14:42:51.224375 18053 heartbeater.cc:347] Connected to a master server at 127.21.55.126:38641
I20260810 14:42:51.224710 18053 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:51.225409 18053 heartbeater.cc:511] Master 127.21.55.126:38641 requested a full tablet report, sending...
I20260810 14:42:51.226886 17374 ts_manager.cc:194] Registered new tserver with Master: 4d9b91ca7fbd4e4ca30412c6c9d123e0 (127.21.55.67:43165)
I20260810 14:42:51.227722 17374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:41543
I20260810 14:42:51.238513 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:42:51.261310 18055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.245098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33718 (local address 127.21.55.126:38641)
0810 14:42:51.245452 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:51.245462 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:51.245929 (+ 467us) server_negotiation.cc:407] Connection header received
0810 14:42:51.246256 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:51.246261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:51.246368 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:51.246489 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:51.248597 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.249426 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.251865 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.252344 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.252508 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.255800 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:51.255805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:51.255808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:51.255842 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:51.258425 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:51.259206 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:51.259211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:51.259213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:51.259333 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:51.259743 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:51.259747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:51.259749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:51.260078 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:51.260348 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:51.260724 (+ 376us) server_negotiation.cc:299] Negotiation successful
0810 14:42:51.260972 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260810 14:42:51.266130 17374 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:42:51.275547 17374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:42:51.320777 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.290775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42986 (local address 127.21.55.67:43165)
0810 14:42:51.306681 (+ 15906us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:51.306693 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:51.309025 (+ 2332us) server_negotiation.cc:407] Connection header received
0810 14:42:51.309544 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:51.309553 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:51.309822 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:51.310023 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:51.311151 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.312958 (+ 1807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.317453 (+ 4495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.319086 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.319192 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.319443 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:51.320169 (+ 726us) server_negotiation.cc:299] Negotiation successful
0810 14:42:51.320383 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15672,"thread_start_us":15528,"threads_started":1}
I20260810 14:42:51.322057 18082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.290148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:43165 (local address 127.0.0.1:42986)
0810 14:42:51.308895 (+ 18747us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:51.308916 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:51.309277 (+ 361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:51.310430 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:51.310438 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:51.310479 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:51.310952 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.310964 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.313154 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.313160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.315847 (+ 2687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.315863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.319646 (+ 3783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.319653 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.319871 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.319930 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 14:42:51.321084 (+ 1154us) client_negotiation.cc:240] Negotiation successful
0810 14:42:51.321541 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18608,"mutex_wait_us":196,"thread_start_us":18905,"threads_started":1}
I20260810 14:42:51.326397 18080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.289407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:44485 (local address 127.0.0.1:42660)
0810 14:42:51.304630 (+ 15223us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:51.304687 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:51.304935 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:51.317741 (+ 12806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:51.317747 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:51.317778 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:51.318279 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.318296 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.320436 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.320443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.323394 (+ 2951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.323420 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.325483 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.325490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.325683 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.325716 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:42:51.325840 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 14:42:51.325977 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14856,"mutex_wait_us":89,"thread_start_us":15786,"threads_started":1}
I20260810 14:42:51.326455 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.289930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42660 (local address 127.21.55.65:44485)
0810 14:42:51.316518 (+ 26588us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:51.316528 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:51.316559 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:42:51.316657 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:51.316665 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:51.316935 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:51.317151 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:51.318556 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.320244 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.323629 (+ 3385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.325230 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.325418 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.325655 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:51.325903 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 14:42:51.326078 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26367,"thread_start_us":26170,"threads_started":1}
I20260810 14:42:51.326798 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.312544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:38487 (local address 127.0.0.1:36484)
0810 14:42:51.316978 (+ 4434us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:51.317005 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:51.317188 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:51.318633 (+ 1445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:51.318640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:51.318670 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:51.319157 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.319172 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.321082 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.321087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.323570 (+ 2483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.323585 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.325849 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.325856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.326048 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.326083 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 14:42:51.326177 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 14:42:51.326308 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4251,"mutex_wait_us":3,"thread_start_us":4071,"threads_started":1}
I20260810 14:42:51.327586 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.306592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36484 (local address 127.21.55.66:38487)
0810 14:42:51.317759 (+ 11167us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:51.317767 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:51.317796 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:42:51.317904 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:51.317914 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:51.318135 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:51.318355 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:51.319378 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.320896 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.323772 (+ 2876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.325305 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.325450 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.325630 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:51.327054 (+ 1424us) server_negotiation.cc:299] Negotiation successful
0810 14:42:51.327232 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10994,"thread_start_us":10877,"threads_started":1}
I20260810 14:42:51.326699 17967 tablet_service.cc:1467] Processing CreateTablet for tablet 263d712818c345b2b449f2e24bfe03de (DEFAULT_TABLE table=test-table [id=7acca83d3eb34d13b850a00dce115803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:51.329267 17967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 263d712818c345b2b449f2e24bfe03de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:51.330389 17571 tablet_service.cc:1467] Processing CreateTablet for tablet 263d712818c345b2b449f2e24bfe03de (DEFAULT_TABLE table=test-table [id=7acca83d3eb34d13b850a00dce115803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:51.331034 17773 tablet_service.cc:1467] Processing CreateTablet for tablet 263d712818c345b2b449f2e24bfe03de (DEFAULT_TABLE table=test-table [id=7acca83d3eb34d13b850a00dce115803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:51.332320 17773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 263d712818c345b2b449f2e24bfe03de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:51.332584 17571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 263d712818c345b2b449f2e24bfe03de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:51.346050 18089 tablet_bootstrap.cc:492] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: Bootstrap starting.
I20260810 14:42:51.350893 18091 tablet_bootstrap.cc:492] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: Bootstrap starting.
I20260810 14:42:51.351649 18093 tablet_bootstrap.cc:492] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: Bootstrap starting.
I20260810 14:42:51.353247 18089 tablet_bootstrap.cc:654] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:51.354485 18091 tablet_bootstrap.cc:654] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:51.354660 18089 log.cc:824] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:51.355597 18093 tablet_bootstrap.cc:654] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:51.355904 18091 log.cc:824] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:51.357517 18093 log.cc:824] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:51.361361 18093 tablet_bootstrap.cc:492] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: No bootstrap required, opened a new log
I20260810 14:42:51.361475 18089 tablet_bootstrap.cc:492] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: No bootstrap required, opened a new log
I20260810 14:42:51.361611 18091 tablet_bootstrap.cc:492] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: No bootstrap required, opened a new log
I20260810 14:42:51.361728 18093 ts_tablet_manager.cc:1397] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 14:42:51.361759 18089 ts_tablet_manager.cc:1397] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.003s
I20260810 14:42:51.361956 18091 ts_tablet_manager.cc:1397] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260810 14:42:51.368906 18093 raft_consensus.cc:348] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22049959f11d4ebeacfb795bf2862c55" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 44485 } } peers { permanent_uuid: "31f23268678248bdaffc01a256aca164" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38487 } } peers { permanent_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43165 } }
I20260810 14:42:51.369313 18093 raft_consensus.cc:374] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:51.369405 18093 raft_consensus.cc:729] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f23268678248bdaffc01a256aca164, State: Initialized, Role: FOLLOWER
I20260810 14:42:51.370052 18093 consensus_queue.cc:260] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22049959f11d4ebeacfb795bf2862c55" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 44485 } } peers { permanent_uuid: "31f23268678248bdaffc01a256aca164" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38487 } } peers { permanent_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43165 } }
I20260810 14:42:51.371209 17878 heartbeater.cc:503] Master 127.21.55.126:38641 was elected leader, sending a full tablet report...
I20260810 14:42:51.372805 18091 raft_consensus.cc:348] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22049959f11d4ebeacfb795bf2862c55" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 44485 } } peers { permanent_uuid: "31f23268678248bdaffc01a256aca164" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38487 } } peers { permanent_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43165 } }
I20260810 14:42:51.372805 18089 raft_consensus.cc:348] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22049959f11d4ebeacfb795bf2862c55" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 44485 } } peers { permanent_uuid: "31f23268678248bdaffc01a256aca164" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38487 } } peers { permanent_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43165 } }
I20260810 14:42:51.373181 18093 ts_tablet_manager.cc:1428] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260810 14:42:51.373239 18091 raft_consensus.cc:374] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:51.373239 18089 raft_consensus.cc:374] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:51.373337 18089 raft_consensus.cc:729] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9b91ca7fbd4e4ca30412c6c9d123e0, State: Initialized, Role: FOLLOWER
I20260810 14:42:51.373337 18091 raft_consensus.cc:729] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22049959f11d4ebeacfb795bf2862c55, State: Initialized, Role: FOLLOWER
I20260810 14:42:51.373934 18091 consensus_queue.cc:260] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22049959f11d4ebeacfb795bf2862c55" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 44485 } } peers { permanent_uuid: "31f23268678248bdaffc01a256aca164" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38487 } } peers { permanent_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43165 } }
I20260810 14:42:51.373926 18089 consensus_queue.cc:260] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22049959f11d4ebeacfb795bf2862c55" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 44485 } } peers { permanent_uuid: "31f23268678248bdaffc01a256aca164" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38487 } } peers { permanent_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43165 } }
I20260810 14:42:51.375554 18053 heartbeater.cc:503] Master 127.21.55.126:38641 was elected leader, sending a full tablet report...
I20260810 14:42:51.375758 18091 ts_tablet_manager.cc:1428] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260810 14:42:51.376060 18089 ts_tablet_manager.cc:1428] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
W20260810 14:42:51.452487 18054 tablet.cc:2369] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:42:51.526026 17668 tablet.cc:2369] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:42:51.597818 17880 tablet.cc:2369] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:42:51.629571 18099 raft_consensus.cc:482] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:42:51.629932 18099 raft_consensus.cc:504] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22049959f11d4ebeacfb795bf2862c55" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 44485 } } peers { permanent_uuid: "31f23268678248bdaffc01a256aca164" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38487 } } peers { permanent_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43165 } }
I20260810 14:42:51.632050 18099 leader_election.cc:288] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31f23268678248bdaffc01a256aca164 (127.21.55.66:38487), 4d9b91ca7fbd4e4ca30412c6c9d123e0 (127.21.55.67:43165)
I20260810 14:42:51.639292 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.632618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:43585 (local address 127.21.55.66:38487)
0810 14:42:51.633103 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:51.633112 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:51.633666 (+ 554us) server_negotiation.cc:407] Connection header received
0810 14:42:51.633866 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:51.633870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:51.633927 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:51.634027 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:51.634607 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.635283 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.637298 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.638270 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.638453 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.638537 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:51.638868 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 14:42:51.638995 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304}
I20260810 14:42:51.639329 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.632521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:55063 (local address 127.21.55.67:43165)
0810 14:42:51.632884 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:51.632894 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:51.633319 (+ 425us) server_negotiation.cc:407] Connection header received
0810 14:42:51.633577 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:51.633581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:51.633641 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:51.633758 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:51.634296 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.635063 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.637411 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.638329 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.638513 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.638613 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:51.638920 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 14:42:51.639066 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
I20260810 14:42:51.639469 18103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.632359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:43165 (local address 127.21.55.65:55063)
0810 14:42:51.633175 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:51.633207 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:51.633406 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:51.633777 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:51.633782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:51.633810 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:51.634145 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.634155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.635191 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.635198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.637249 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.637263 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.638605 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.638612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.638763 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.638787 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 14:42:51.638895 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 14:42:51.639078 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":85,"thread_start_us":460,"threads_started":1}
I20260810 14:42:51.639833 18105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.632560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:38487 (local address 127.21.55.65:43585)
0810 14:42:51.633569 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:51.633582 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:51.633711 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:51.634051 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:51.634055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:51.634081 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:51.634443 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.634451 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.635431 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.635442 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.637145 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:51.637158 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.638510 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.638517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:51.638697 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.638722 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 14:42:51.638832 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 14:42:51.638984 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":26,"thread_start_us":656,"threads_started":1}
I20260810 14:42:51.640463 17990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "263d712818c345b2b449f2e24bfe03de" candidate_uuid: "22049959f11d4ebeacfb795bf2862c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" is_pre_election: true
I20260810 14:42:51.640928 17990 raft_consensus.cc:2454] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22049959f11d4ebeacfb795bf2862c55 in term 0.
I20260810 14:42:51.640838 17802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "263d712818c345b2b449f2e24bfe03de" candidate_uuid: "22049959f11d4ebeacfb795bf2862c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f23268678248bdaffc01a256aca164" is_pre_election: true
I20260810 14:42:51.641369 17802 raft_consensus.cc:2454] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22049959f11d4ebeacfb795bf2862c55 in term 0.
I20260810 14:42:51.641625 17504 leader_election.cc:302] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22049959f11d4ebeacfb795bf2862c55, 4d9b91ca7fbd4e4ca30412c6c9d123e0; no voters:
I20260810 14:42:51.642001 18099 raft_consensus.cc:2781] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:42:51.642139 18099 raft_consensus.cc:482] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:42:51.642242 18099 raft_consensus.cc:3037] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:51.644490 18099 raft_consensus.cc:504] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22049959f11d4ebeacfb795bf2862c55" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 44485 } } peers { permanent_uuid: "31f23268678248bdaffc01a256aca164" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38487 } } peers { permanent_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43165 } }
I20260810 14:42:51.645129 18099 leader_election.cc:288] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [CANDIDATE]: Term 1 election: Requested vote from peers 31f23268678248bdaffc01a256aca164 (127.21.55.66:38487), 4d9b91ca7fbd4e4ca30412c6c9d123e0 (127.21.55.67:43165)
I20260810 14:42:51.645648 17802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "263d712818c345b2b449f2e24bfe03de" candidate_uuid: "22049959f11d4ebeacfb795bf2862c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f23268678248bdaffc01a256aca164"
I20260810 14:42:51.645781 17990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "263d712818c345b2b449f2e24bfe03de" candidate_uuid: "22049959f11d4ebeacfb795bf2862c55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0"
I20260810 14:42:51.645857 17802 raft_consensus.cc:3037] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:51.646093 17990 raft_consensus.cc:3037] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:51.648491 17802 raft_consensus.cc:2454] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22049959f11d4ebeacfb795bf2862c55 in term 1.
I20260810 14:42:51.648803 17990 raft_consensus.cc:2454] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22049959f11d4ebeacfb795bf2862c55 in term 1.
I20260810 14:42:51.648943 17502 leader_election.cc:302] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22049959f11d4ebeacfb795bf2862c55, 31f23268678248bdaffc01a256aca164; no voters:
I20260810 14:42:51.649266 18099 raft_consensus.cc:2781] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:42:51.649756 18099 raft_consensus.cc:686] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 1 LEADER]: Becoming Leader. State: Replica: 22049959f11d4ebeacfb795bf2862c55, State: Running, Role: LEADER
I20260810 14:42:51.650315 18099 consensus_queue.cc:237] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22049959f11d4ebeacfb795bf2862c55" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 44485 } } peers { permanent_uuid: "31f23268678248bdaffc01a256aca164" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38487 } } peers { permanent_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43165 } }
I20260810 14:42:51.654902 17374 catalog_manager.cc:5495] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22049959f11d4ebeacfb795bf2862c55 (127.21.55.65). New cstate: current_term: 1 leader_uuid: "22049959f11d4ebeacfb795bf2862c55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22049959f11d4ebeacfb795bf2862c55" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 44485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31f23268678248bdaffc01a256aca164" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43165 } health_report { overall_health: UNKNOWN } } }
I20260810 14:42:51.678803 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.673219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42672 (local address 127.21.55.65:44485)
0810 14:42:51.673409 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:51.673414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:51.673512 (+ 98us) server_negotiation.cc:407] Connection header received
0810 14:42:51.673700 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:51.673704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:51.673783 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:51.673914 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:42:51.674594 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.675325 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.676704 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.677102 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.677198 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.677727 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:42:51.678209 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:42:51.678316 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:51.678418 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 14:42:51.678483 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:42:51.686730 17990 raft_consensus.cc:1261] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0 [term 1 FOLLOWER]: Refusing update from remote peer 22049959f11d4ebeacfb795bf2862c55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:42:51.687172 17802 raft_consensus.cc:1261] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164 [term 1 FOLLOWER]: Refusing update from remote peer 22049959f11d4ebeacfb795bf2862c55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:42:51.687793 18106 consensus_queue.cc:1035] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9b91ca7fbd4e4ca30412c6c9d123e0" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:42:51.688277 18099 consensus_queue.cc:1035] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31f23268678248bdaffc01a256aca164" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:42:51.717147 17374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:42:51.717480 17374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:42:51.721511 17374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27c88dc7550f429686c50063ee111529: Sending DeleteTablet for 3 replicas of tablet 263d712818c345b2b449f2e24bfe03de
I20260810 14:42:51.722826 17571 tablet_service.cc:1514] Processing DeleteTablet for tablet 263d712818c345b2b449f2e24bfe03de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:51 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:42660
I20260810 14:42:51.722872 17967 tablet_service.cc:1514] Processing DeleteTablet for tablet 263d712818c345b2b449f2e24bfe03de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:51 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:42986
I20260810 14:42:51.722939 17773 tablet_service.cc:1514] Processing DeleteTablet for tablet 263d712818c345b2b449f2e24bfe03de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:51 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:36484
I20260810 14:42:51.729197 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 17473
I20260810 14:42:51.735105 18127 tablet_replica.cc:330] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: stopping tablet replica
I20260810 14:42:51.735806 18130 tablet_replica.cc:330] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: stopping tablet replica
I20260810 14:42:51.736081 18127 raft_consensus.cc:2229] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:42:51.736642 18130 raft_consensus.cc:2229] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:42:51.741632 18130 raft_consensus.cc:2258] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:42:51.744817 18130 ts_tablet_manager.cc:1905] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:51.746133 18127 raft_consensus.cc:2258] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 10 14:42:51 dist-test-slave-h6n0 krb5kdc[17302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:42:51.747299 18128 tablet_replica.cc:330] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: stopping tablet replica
I20260810 14:42:51.747785 18128 raft_consensus.cc:2229] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:42:51.748409 18128 raft_consensus.cc:2258] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:42:51.749573 18127 ts_tablet_manager.cc:1905] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:51.753103 18128 ts_tablet_manager.cc:1905] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:51.754410 18130 ts_tablet_manager.cc:1918] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:42:51.754679 18130 log.cc:1196] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/263d712818c345b2b449f2e24bfe03de
I20260810 14:42:51.755307 18130 ts_tablet_manager.cc:1939] T 263d712818c345b2b449f2e24bfe03de P 22049959f11d4ebeacfb795bf2862c55: Deleting consensus metadata
I20260810 14:42:51.756982 18127 ts_tablet_manager.cc:1918] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:42:51.757157 18127 log.cc:1196] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/263d712818c345b2b449f2e24bfe03de
I20260810 14:42:51.757246 17353 catalog_manager.cc:4841] TS 22049959f11d4ebeacfb795bf2862c55 (127.21.55.65:44485): tablet 263d712818c345b2b449f2e24bfe03de (table test-table [id=7acca83d3eb34d13b850a00dce115803]) successfully deleted
I20260810 14:42:51.757710 18127 ts_tablet_manager.cc:1939] T 263d712818c345b2b449f2e24bfe03de P 4d9b91ca7fbd4e4ca30412c6c9d123e0: Deleting consensus metadata
I20260810 14:42:51.759596 17356 catalog_manager.cc:4841] TS 4d9b91ca7fbd4e4ca30412c6c9d123e0 (127.21.55.67:43165): tablet 263d712818c345b2b449f2e24bfe03de (table test-table [id=7acca83d3eb34d13b850a00dce115803]) successfully deleted
I20260810 14:42:51.760087 18128 ts_tablet_manager.cc:1918] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:42:51.760269 18128 log.cc:1196] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/263d712818c345b2b449f2e24bfe03de
I20260810 14:42:51.760864 18128 ts_tablet_manager.cc:1939] T 263d712818c345b2b449f2e24bfe03de P 31f23268678248bdaffc01a256aca164: Deleting consensus metadata
I20260810 14:42:51.760977 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.732276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.21.55.65:44485)
0810 14:42:51.732734 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:51.732741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:51.732763 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:42:51.732826 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:51.732832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:51.732922 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:51.733028 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:51.736420 (+ 3392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.738729 (+ 2309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.741339 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.742138 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.742284 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.748872 (+ 6588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:51.748887 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:51.748905 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:51.748962 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:51.754656 (+ 5694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:51.758064 (+ 3408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:51.758077 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:51.758080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:51.758265 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:51.758842 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:51.758848 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:51.758851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:51.759335 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:51.759697 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:51.760302 (+ 605us) server_negotiation.cc:299] Negotiation successful
0810 14:42:51.760565 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314}
I20260810 14:42:51.761736 17657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:51.762600 17353 catalog_manager.cc:4841] TS 31f23268678248bdaffc01a256aca164 (127.21.55.66:38487): tablet 263d712818c345b2b449f2e24bfe03de (table test-table [id=7acca83d3eb34d13b850a00dce115803]) successfully deleted
I20260810 14:42:51.864853 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 17473
I20260810 14:42:51.895218 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 17674
Aug 10 14:42:51 dist-test-slave-h6n0 krb5kdc[17302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:42:51.911424 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:51.896083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36494 (local address 127.21.55.66:38487)
0810 14:42:51.896296 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:51.896305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:51.896386 (+ 81us) server_negotiation.cc:407] Connection header received
0810 14:42:51.896597 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:51.896601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:51.896665 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:51.896784 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:51.898101 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.899105 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.900374 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:51.900770 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:51.900897 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:51.905433 (+ 4536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:51.905447 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:51.905463 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:51.905496 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:51.908570 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:51.909285 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:51.909293 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:51.909295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:51.909414 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:51.909791 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:51.909796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:51.909799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:51.910232 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:51.910574 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:51.910853 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 14:42:51.911079 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 14:42:51.912115 17869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:52.003209 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 17674
I20260810 14:42:52.027988 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 17886
Aug 10 14:42:52 dist-test-slave-h6n0 krb5kdc[17302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:42:52.047132 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:52.028823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42990 (local address 127.21.55.67:43165)
0810 14:42:52.029037 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:52.029043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:52.029065 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:42:52.029197 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:52.029202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:52.029268 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:52.029387 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:52.030940 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:52.032276 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:52.034855 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:52.035503 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:52.035699 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:52.040349 (+ 4650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:52.040362 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:52.040378 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:52.040416 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:52.043402 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:52.044171 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:52.044179 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:52.044181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:52.044447 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:52.044966 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:52.044972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:52.044976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:52.045378 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:52.045729 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:52.046135 (+ 406us) server_negotiation.cc:299] Negotiation successful
0810 14:42:52.046447 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 14:42:52.048166 18047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:52.142865 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 17886
I20260810 14:42:52.168540 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 17320
I20260810 14:42:52.169809 17426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:52.287155 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 17320
2026-08-10T14:42:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5259 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T14:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:42:52Z Disabled control of system clock
I20260810 14:42:52.400825 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:42415
--webserver_interface=127.21.55.126
--webserver_port=0
--builtin_ntp_servers=127.21.55.84:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:42415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:42:52.759904 18141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:52.760245 18141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:52.760334 18141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:52.769709 18141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:42:52.769896 18141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:52.769939 18141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:52.769999 18141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:42:52.770054 18141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:42:52.781278 18141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:42415
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:42415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:52.782668 18141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:52.784488 18141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:52.797884 18147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:52.797932 18148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:52.797842 18150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:52.804534 18141 server_base.cc:1034] running on GCE node
I20260810 14:42:52.805462 18141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:52.807654 18141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:52.808898 18141 hybrid_clock.cc:648] HybridClock initialized: now 1786372972808767 us; error 127 us; skew 500 ppm
I20260810 14:42:52.809319 18141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:42:52.811841 18141 webserver.cc:460] Webserver started at http://127.21.55.126:40095/ using document root <none> and password file <none>
I20260810 14:42:52.812577 18141 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:52.812678 18141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:52.813012 18141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:52.816138 18141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "c8b4c44f97d74b36bf9f4432cf74f20d"
format_stamp: "Formatted at 2026-08-10 14:42:52 on dist-test-slave-h6n0"
server_key: "870f854e37b1c95bb5b6a2265052f398"
server_key_iv: "87640bb838a4cc54856bcc12337ad087"
server_key_version: "encryptionkey@0"
I20260810 14:42:52.817004 18141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "c8b4c44f97d74b36bf9f4432cf74f20d"
format_stamp: "Formatted at 2026-08-10 14:42:52 on dist-test-slave-h6n0"
server_key: "870f854e37b1c95bb5b6a2265052f398"
server_key_iv: "87640bb838a4cc54856bcc12337ad087"
server_key_version: "encryptionkey@0"
I20260810 14:42:52.822259 18141 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260810 14:42:52.825940 18158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:52.827574 18141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:42:52.827781 18141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "c8b4c44f97d74b36bf9f4432cf74f20d"
format_stamp: "Formatted at 2026-08-10 14:42:52 on dist-test-slave-h6n0"
server_key: "870f854e37b1c95bb5b6a2265052f398"
server_key_iv: "87640bb838a4cc54856bcc12337ad087"
server_key_version: "encryptionkey@0"
I20260810 14:42:52.828007 18141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:52.847715 18141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:52.848987 18141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:52.849376 18141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:52.872144 18141 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:42415
I20260810 14:42:52.872169 18232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:42415 every 8 connection(s)
I20260810 14:42:52.874089 18141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:42:52.877942 18233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:52.881573 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 18141
I20260810 14:42:52.881958 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:42:52.882635 21725 external_mini_cluster.cc:1428] Setting key ad25af641d9be3719f9c880c7a78d9b2
I20260810 14:42:52.889246 18233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d: Bootstrap starting.
I20260810 14:42:52.892940 18233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:52.894217 18233 log.cc:824] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:52.897655 18233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d: No bootstrap required, opened a new log
I20260810 14:42:52.899535 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:52.884855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.21.55.126:42415)
0810 14:42:52.885859 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:52.885869 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:52.885916 (+ 47us) server_negotiation.cc:407] Connection header received
0810 14:42:52.888348 (+ 2432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:52.888391 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:52.889082 (+ 691us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:52.889639 (+ 557us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:52.890293 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:52.892134 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:52.894171 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:52.895450 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:52.895655 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:52.897166 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:52.897177 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:52.897191 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:42:52.897773 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:52.897868 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:42:52.897890 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:52.897983 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:52.898269 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 14:42:52.898632 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":405,"threads_started":1}
I20260810 14:42:52.908236 18233 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4c44f97d74b36bf9f4432cf74f20d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 42415 } }
I20260810 14:42:52.908918 18233 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:52.909061 18233 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b4c44f97d74b36bf9f4432cf74f20d, State: Initialized, Role: FOLLOWER
I20260810 14:42:52.909816 18233 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4c44f97d74b36bf9f4432cf74f20d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 42415 } }
I20260810 14:42:52.910089 18233 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:42:52.910257 18233 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:42:52.910445 18233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:52.913106 18233 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4c44f97d74b36bf9f4432cf74f20d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 42415 } }
I20260810 14:42:52.913719 18233 leader_election.cc:302] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8b4c44f97d74b36bf9f4432cf74f20d; no voters:
I20260810 14:42:52.914333 18233 leader_election.cc:288] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:42:52.914595 18241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:42:52.915349 18241 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [term 1 LEADER]: Becoming Leader. State: Replica: c8b4c44f97d74b36bf9f4432cf74f20d, State: Running, Role: LEADER
I20260810 14:42:52.915890 18241 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4c44f97d74b36bf9f4432cf74f20d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 42415 } }
I20260810 14:42:52.916608 18233 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:42:52.919740 18243 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8b4c44f97d74b36bf9f4432cf74f20d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4c44f97d74b36bf9f4432cf74f20d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 42415 } } }
I20260810 14:42:52.920153 18243 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [sys.catalog]: This master's current role is: LEADER
I20260810 14:42:52.922722 18244 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8b4c44f97d74b36bf9f4432cf74f20d. Latest consensus state: current_term: 1 leader_uuid: "c8b4c44f97d74b36bf9f4432cf74f20d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4c44f97d74b36bf9f4432cf74f20d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 42415 } } }
I20260810 14:42:52.923115 18244 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d [sys.catalog]: This master's current role is: LEADER
I20260810 14:42:52.926388 18252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:42:52.930891 18252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:42:52.940732 18252 catalog_manager.cc:1347] Generated new cluster ID: 108cdf7d386f4ecaa80bba17c229bc08
I20260810 14:42:52.940850 18252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:42:52.963069 18252 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:42:52.965513 18252 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:42:52.979970 18252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d: Generated new TSK 0
I20260810 14:42:52.981433 18252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:42:53.002430 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:42415
--builtin_ntp_servers=127.21.55.84:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:42:53.371778 18271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:53.372061 18271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:53.372196 18271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:53.379557 18271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:53.379653 18271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:53.379797 18271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:42:53.388936 18271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:42415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:53.390202 18271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:53.392138 18271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:53.406229 18278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:53.406546 18279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:53.406574 18281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:53.407617 18271 server_base.cc:1034] running on GCE node
I20260810 14:42:53.408315 18271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:53.409256 18271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:53.410517 18271 hybrid_clock.cc:648] HybridClock initialized: now 1786372973410485 us; error 51 us; skew 500 ppm
I20260810 14:42:53.410943 18271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:42:53.413631 18271 webserver.cc:460] Webserver started at http://127.21.55.65:37141/ using document root <none> and password file <none>
I20260810 14:42:53.414609 18271 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:53.414747 18271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:53.415091 18271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:53.418072 18271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "35fe530eeeb348aa8f1554588bda5e4f"
format_stamp: "Formatted at 2026-08-10 14:42:53 on dist-test-slave-h6n0"
server_key: "6dac2194e753095b9729e215876e7bce"
server_key_iv: "35594a2b1435695a97eb78b8e54534fb"
server_key_version: "encryptionkey@0"
I20260810 14:42:53.419355 18271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "35fe530eeeb348aa8f1554588bda5e4f"
format_stamp: "Formatted at 2026-08-10 14:42:53 on dist-test-slave-h6n0"
server_key: "6dac2194e753095b9729e215876e7bce"
server_key_iv: "35594a2b1435695a97eb78b8e54534fb"
server_key_version: "encryptionkey@0"
I20260810 14:42:53.424748 18271 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260810 14:42:53.429041 18299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:53.430442 18271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:42:53.430709 18271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "35fe530eeeb348aa8f1554588bda5e4f"
format_stamp: "Formatted at 2026-08-10 14:42:53 on dist-test-slave-h6n0"
server_key: "6dac2194e753095b9729e215876e7bce"
server_key_iv: "35594a2b1435695a97eb78b8e54534fb"
server_key_version: "encryptionkey@0"
I20260810 14:42:53.430996 18271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:53.453469 18271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:53.454470 18271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:53.454931 18271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:53.456248 18271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:53.458576 18271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:53.458752 18271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 14:42:53.458935 18271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:53.459045 18271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:53.500339 18271 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:35485
I20260810 14:42:53.500437 18463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:35485 every 8 connection(s)
I20260810 14:42:53.501713 18271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:42:53.506482 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 18271
I20260810 14:42:53.506707 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:42:53.507273 21725 external_mini_cluster.cc:1428] Setting key 47860bbecd792371bd03c83fad4451e4
I20260810 14:42:53.510843 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:42415
--builtin_ntp_servers=127.21.55.84:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:42:53.517628 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:53.504457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:39495 (local address 127.21.55.126:42415)
0810 14:42:53.504950 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:53.504954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:53.506997 (+ 2043us) server_negotiation.cc:407] Connection header received
0810 14:42:53.510090 (+ 3093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:53.510096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:53.510176 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:53.510295 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:53.511552 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:53.512754 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:53.514364 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:53.515101 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:53.515241 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:53.516624 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:53.516633 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:53.516640 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:42:53.516666 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:53.516721 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:42:53.516741 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:53.516835 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:53.517041 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 14:42:53.517151 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":263,"threads_started":1}
I20260810 14:42:53.518664 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:53.505147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:42415 (local address 127.21.55.65:39495)
0810 14:42:53.506748 (+ 1601us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:53.506834 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:53.509791 (+ 2957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:53.510491 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:53.510499 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:53.510671 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:53.511372 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:53.511391 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:53.512934 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:53.512938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:53.514216 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:53.514230 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:53.515321 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:53.515327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:53.515524 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:53.516293 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:42:53.516296 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:53.516336 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:42:53.516419 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:53.516840 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:53.516847 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:53.516853 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:42:53.516995 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 14:42:53.517399 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":403,"threads_started":1}
I20260810 14:42:53.520203 18464 heartbeater.cc:347] Connected to a master server at 127.21.55.126:42415
I20260810 14:42:53.520543 18464 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:53.521793 18464 heartbeater.cc:511] Master 127.21.55.126:42415 requested a full tablet report, sending...
I20260810 14:42:53.524784 18185 ts_manager.cc:194] Registered new tserver with Master: 35fe530eeeb348aa8f1554588bda5e4f (127.21.55.65:35485)
I20260810 14:42:53.526676 18185 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.65:39495
W20260810 14:42:53.890239 18470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:53.890607 18470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:53.890885 18470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:53.898193 18470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:53.898334 18470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:53.898514 18470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:42:53.907655 18470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:42415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:53.908950 18470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:53.910796 18470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:53.925228 18483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:53.925208 18481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:53.925355 18485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:53.926925 18470 server_base.cc:1034] running on GCE node
I20260810 14:42:53.927606 18470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:53.928583 18470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:53.929962 18470 hybrid_clock.cc:648] HybridClock initialized: now 1786372973929789 us; error 60 us; skew 500 ppm
I20260810 14:42:53.930516 18470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:42:53.933298 18470 webserver.cc:460] Webserver started at http://127.21.55.66:40145/ using document root <none> and password file <none>
I20260810 14:42:53.934046 18470 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:53.934186 18470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:53.934648 18470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:53.937536 18470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "b0d6e0f4e9a34456b3ba93034e561910"
format_stamp: "Formatted at 2026-08-10 14:42:53 on dist-test-slave-h6n0"
server_key: "6ec51b40af5c4e381c63d82e5a8afa0c"
server_key_iv: "17ef526a51498c3f3cb0f990ffc23422"
server_key_version: "encryptionkey@0"
I20260810 14:42:53.938344 18470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "b0d6e0f4e9a34456b3ba93034e561910"
format_stamp: "Formatted at 2026-08-10 14:42:53 on dist-test-slave-h6n0"
server_key: "6ec51b40af5c4e381c63d82e5a8afa0c"
server_key_iv: "17ef526a51498c3f3cb0f990ffc23422"
server_key_version: "encryptionkey@0"
I20260810 14:42:53.944124 18470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:42:53.948920 18493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:53.950788 18470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:42:53.951013 18470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "b0d6e0f4e9a34456b3ba93034e561910"
format_stamp: "Formatted at 2026-08-10 14:42:53 on dist-test-slave-h6n0"
server_key: "6ec51b40af5c4e381c63d82e5a8afa0c"
server_key_iv: "17ef526a51498c3f3cb0f990ffc23422"
server_key_version: "encryptionkey@0"
I20260810 14:42:53.951280 18470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:53.982260 18470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:53.983152 18470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:53.983425 18470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:53.984437 18470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:53.985939 18470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:53.986033 18470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:53.986123 18470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:53.986189 18470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:54.023665 18470 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:34741
I20260810 14:42:54.023757 18662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:34741 every 8 connection(s)
I20260810 14:42:54.025007 18470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:42:54.026299 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 18470
I20260810 14:42:54.026517 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:42:54.027139 21725 external_mini_cluster.cc:1428] Setting key 44ef316a857664123649f20470a0d026
I20260810 14:42:54.030762 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:42415
--builtin_ntp_servers=127.21.55.84:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:42:54.039754 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.028243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:36167 (local address 127.21.55.126:42415)
0810 14:42:54.028793 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:54.028798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:54.030251 (+ 1453us) server_negotiation.cc:407] Connection header received
0810 14:42:54.032333 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:54.032342 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:54.032431 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:54.032583 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:54.033665 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.034851 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.036533 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.037255 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.037397 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.038735 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:54.038744 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:54.038751 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:42:54.038776 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:54.038831 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:42:54.038850 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:54.038948 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:54.039220 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 14:42:54.039329 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":291,"threads_started":1}
I20260810 14:42:54.040566 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.029182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:42415 (local address 127.21.55.66:36167)
0810 14:42:54.030105 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:54.030144 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:54.032045 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:54.032815 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:54.032827 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:54.032923 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:54.033482 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.033498 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.035003 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.035007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.036386 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.036399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.037464 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.037469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.037670 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.038388 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:42:54.038391 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:54.038428 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:42:54.038499 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:54.038999 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:54.039009 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:54.039016 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 14:42:54.039177 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 14:42:54.039460 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":321,"threads_started":1}
I20260810 14:42:54.042018 18663 heartbeater.cc:347] Connected to a master server at 127.21.55.126:42415
I20260810 14:42:54.042320 18663 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:54.043118 18663 heartbeater.cc:511] Master 127.21.55.126:42415 requested a full tablet report, sending...
I20260810 14:42:54.044628 18185 ts_manager.cc:194] Registered new tserver with Master: b0d6e0f4e9a34456b3ba93034e561910 (127.21.55.66:34741)
I20260810 14:42:54.045869 18185 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.66:36167
W20260810 14:42:54.382265 18672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:54.382587 18672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:54.382725 18672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:54.389767 18672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:54.389853 18672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:54.390036 18672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:42:54.399041 18672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:42415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:54.400206 18672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:54.401859 18672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:54.415103 18678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:54.415079 18680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:54.415712 18683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:54.416261 18672 server_base.cc:1034] running on GCE node
I20260810 14:42:54.416873 18672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:54.417822 18672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:54.419047 18672 hybrid_clock.cc:648] HybridClock initialized: now 1786372974418986 us; error 65 us; skew 500 ppm
I20260810 14:42:54.419364 18672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:42:54.425499 18672 webserver.cc:460] Webserver started at http://127.21.55.67:42709/ using document root <none> and password file <none>
I20260810 14:42:54.426189 18672 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:54.426314 18672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:54.426712 18672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:54.429462 18672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "d3376673d7694334ab808f3d47552e87"
format_stamp: "Formatted at 2026-08-10 14:42:54 on dist-test-slave-h6n0"
server_key: "f8b701c00b7e45ecdde2c96fb5c273b9"
server_key_iv: "e9d0bb70870a0e577ba8c2e4d4231bf9"
server_key_version: "encryptionkey@0"
I20260810 14:42:54.430240 18672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "d3376673d7694334ab808f3d47552e87"
format_stamp: "Formatted at 2026-08-10 14:42:54 on dist-test-slave-h6n0"
server_key: "f8b701c00b7e45ecdde2c96fb5c273b9"
server_key_iv: "e9d0bb70870a0e577ba8c2e4d4231bf9"
server_key_version: "encryptionkey@0"
I20260810 14:42:54.435539 18672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260810 14:42:54.439112 18691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:54.440369 18672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:42:54.440555 18672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "d3376673d7694334ab808f3d47552e87"
format_stamp: "Formatted at 2026-08-10 14:42:54 on dist-test-slave-h6n0"
server_key: "f8b701c00b7e45ecdde2c96fb5c273b9"
server_key_iv: "e9d0bb70870a0e577ba8c2e4d4231bf9"
server_key_version: "encryptionkey@0"
I20260810 14:42:54.440758 18672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:54.469209 18672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:54.470036 18672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:54.470350 18672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:54.471560 18672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:54.473309 18672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:54.473412 18672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:54.473489 18672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:54.473551 18672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:54.513595 18672 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:45935
I20260810 14:42:54.513701 18861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:45935 every 8 connection(s)
I20260810 14:42:54.514999 18672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:42:54.525249 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 18672
I20260810 14:42:54.525502 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:42:54.526062 21725 external_mini_cluster.cc:1428] Setting key d29d2bea21546fc6f7c8e3459fe85993
I20260810 14:42:54.528764 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.518549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:47487 (local address 127.21.55.126:42415)
0810 14:42:54.518736 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:54.518741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:54.520140 (+ 1399us) server_negotiation.cc:407] Connection header received
0810 14:42:54.521967 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:54.521973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:54.522063 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:54.522185 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:54.523299 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.524047 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.525584 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.526019 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.526151 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.527735 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:54.527740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:54.527743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:42:54.527760 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:54.527791 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:42:54.527803 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:54.527880 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:54.528291 (+ 411us) server_negotiation.cc:299] Negotiation successful
0810 14:42:54.528406 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:42:54.529668 18866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.519061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:42415 (local address 127.21.55.67:47487)
0810 14:42:54.519971 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:54.520019 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:54.521705 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:54.522413 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:54.522421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:54.522501 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:54.523088 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.523152 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.524193 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.524199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.525433 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.525454 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.526254 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.526266 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.526485 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.527335 (+ 850us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:42:54.527339 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:54.527397 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:42:54.527494 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:54.527914 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:54.527925 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:54.527933 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 14:42:54.528159 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 14:42:54.528520 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":298,"threads_started":1}
I20260810 14:42:54.530026 18464 heartbeater.cc:503] Master 127.21.55.126:42415 was elected leader, sending a full tablet report...
I20260810 14:42:54.531591 18863 heartbeater.cc:347] Connected to a master server at 127.21.55.126:42415
I20260810 14:42:54.531957 18863 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:54.532720 18863 heartbeater.cc:511] Master 127.21.55.126:42415 requested a full tablet report, sending...
I20260810 14:42:54.534236 18184 ts_manager.cc:194] Registered new tserver with Master: d3376673d7694334ab808f3d47552e87 (127.21.55.67:45935)
I20260810 14:42:54.535190 18184 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.67:47487
I20260810 14:42:54.542459 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:42:54.553808 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.546750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55584 (local address 127.21.55.126:42415)
0810 14:42:54.546975 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:54.546985 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:54.547573 (+ 588us) server_negotiation.cc:407] Connection header received
0810 14:42:54.547750 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:54.547753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:54.547819 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:54.547914 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:54.548553 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.549280 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.551211 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.551646 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.551746 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.552896 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:54.552902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:54.552906 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:42:54.552932 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:54.552986 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:42:54.553001 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:54.553241 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:54.553354 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 14:42:54.553428 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:42:54.557883 18184 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:42:54.564786 18184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:42:54.608002 18879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.577590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45074 (local address 127.21.55.67:45935)
0810 14:42:54.597681 (+ 20091us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:54.597705 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:54.599164 (+ 1459us) server_negotiation.cc:407] Connection header received
0810 14:42:54.599281 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:54.599292 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:54.599579 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:54.600422 (+ 843us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:54.601072 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.602712 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.605755 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.606747 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.606901 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.607209 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:54.607420 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 14:42:54.607593 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19879,"thread_start_us":19688,"threads_started":1}
I20260810 14:42:54.608815 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.577252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:45935 (local address 127.0.0.1:45074)
0810 14:42:54.598201 (+ 20949us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:54.598252 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:54.598468 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:54.599811 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:54.599819 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:54.599851 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:54.600400 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.600426 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.602869 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.602877 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.605471 (+ 2594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.605483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.606939 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.606944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.607153 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.607215 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 14:42:54.607376 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 14:42:54.608401 (+ 1025us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20629,"mutex_wait_us":170,"spinlock_wait_cycles":1920,"thread_start_us":22029,"threads_started":1}
I20260810 14:42:54.609926 18876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.582746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:34741 (local address 127.0.0.1:38148)
0810 14:42:54.599545 (+ 16799us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:54.599563 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:54.599692 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:54.600285 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:54.600290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:54.600327 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:54.600951 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.600966 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.602921 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.602930 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.605327 (+ 2397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.605364 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.609129 (+ 3765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.609138 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.609327 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.609360 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:42:54.609465 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 14:42:54.609600 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16689,"mutex_wait_us":253,"thread_start_us":15002,"threads_started":1}
I20260810 14:42:54.609915 18880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.578085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43480 (local address 127.21.55.65:35485)
0810 14:42:54.596439 (+ 18354us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:54.596454 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:54.599462 (+ 3008us) server_negotiation.cc:407] Connection header received
0810 14:42:54.599560 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:54.599565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:54.599791 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:54.599991 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:54.600720 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.602301 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.605806 (+ 3505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.607724 (+ 1918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.607881 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.609271 (+ 1390us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:54.609351 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 14:42:54.609560 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18079,"thread_start_us":17886,"threads_started":1}
I20260810 14:42:54.610520 18877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.577392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:35485 (local address 127.0.0.1:43480)
0810 14:42:54.599129 (+ 21737us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:54.599153 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:54.599313 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:54.600009 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:54.600013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:54.600048 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:54.600546 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.600560 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.602429 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.602436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.605465 (+ 3029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.605483 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.608522 (+ 3039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.608529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.608735 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.608767 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:42:54.608858 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 14:42:54.610188 (+ 1330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21565,"mutex_wait_us":216,"spinlock_wait_cycles":1792,"thread_start_us":21365,"threads_started":1}
I20260810 14:42:54.610903 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.582995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38148 (local address 127.21.55.66:34741)
0810 14:42:54.599098 (+ 16103us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:54.599107 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:54.599679 (+ 572us) server_negotiation.cc:407] Connection header received
0810 14:42:54.599841 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:54.599850 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:54.600050 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:54.600251 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:54.601173 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.602766 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.605805 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.607724 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.607889 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.608130 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:54.610295 (+ 2165us) server_negotiation.cc:299] Negotiation successful
0810 14:42:54.610511 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15942,"thread_start_us":15960,"threads_started":1}
I20260810 14:42:54.612468 18764 tablet_service.cc:1467] Processing CreateTablet for tablet 850f511a73fb48b7886372bf272a4e13 (DEFAULT_TABLE table=test-table [id=435590c7ad2b4a67a94a1a79118dc2c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:54.613835 18764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850f511a73fb48b7886372bf272a4e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:54.614145 18563 tablet_service.cc:1467] Processing CreateTablet for tablet 850f511a73fb48b7886372bf272a4e13 (DEFAULT_TABLE table=test-table [id=435590c7ad2b4a67a94a1a79118dc2c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:54.615504 18563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850f511a73fb48b7886372bf272a4e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:54.615396 18366 tablet_service.cc:1467] Processing CreateTablet for tablet 850f511a73fb48b7886372bf272a4e13 (DEFAULT_TABLE table=test-table [id=435590c7ad2b4a67a94a1a79118dc2c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:54.616763 18366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850f511a73fb48b7886372bf272a4e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:54.628844 18883 tablet_bootstrap.cc:492] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: Bootstrap starting.
I20260810 14:42:54.631148 18884 tablet_bootstrap.cc:492] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: Bootstrap starting.
I20260810 14:42:54.632716 18886 tablet_bootstrap.cc:492] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: Bootstrap starting.
I20260810 14:42:54.633540 18883 tablet_bootstrap.cc:654] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:54.634683 18884 tablet_bootstrap.cc:654] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:54.634930 18883 log.cc:824] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:54.635725 18886 tablet_bootstrap.cc:654] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:54.635950 18884 log.cc:824] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:54.636899 18886 log.cc:824] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:54.639189 18884 tablet_bootstrap.cc:492] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: No bootstrap required, opened a new log
I20260810 14:42:54.639456 18884 ts_tablet_manager.cc:1397] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260810 14:42:54.639995 18886 tablet_bootstrap.cc:492] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: No bootstrap required, opened a new log
I20260810 14:42:54.640028 18883 tablet_bootstrap.cc:492] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: No bootstrap required, opened a new log
I20260810 14:42:54.640249 18886 ts_tablet_manager.cc:1397] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 14:42:54.640249 18883 ts_tablet_manager.cc:1397] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.007s
I20260810 14:42:54.648357 18884 raft_consensus.cc:348] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3376673d7694334ab808f3d47552e87" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45935 } } peers { permanent_uuid: "35fe530eeeb348aa8f1554588bda5e4f" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35485 } } peers { permanent_uuid: "b0d6e0f4e9a34456b3ba93034e561910" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34741 } }
I20260810 14:42:54.648739 18884 raft_consensus.cc:374] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:54.648845 18884 raft_consensus.cc:729] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0d6e0f4e9a34456b3ba93034e561910, State: Initialized, Role: FOLLOWER
I20260810 14:42:54.649470 18884 consensus_queue.cc:260] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3376673d7694334ab808f3d47552e87" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45935 } } peers { permanent_uuid: "35fe530eeeb348aa8f1554588bda5e4f" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35485 } } peers { permanent_uuid: "b0d6e0f4e9a34456b3ba93034e561910" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34741 } }
I20260810 14:42:54.650665 18663 heartbeater.cc:503] Master 127.21.55.126:42415 was elected leader, sending a full tablet report...
I20260810 14:42:54.650967 18883 raft_consensus.cc:348] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3376673d7694334ab808f3d47552e87" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45935 } } peers { permanent_uuid: "35fe530eeeb348aa8f1554588bda5e4f" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35485 } } peers { permanent_uuid: "b0d6e0f4e9a34456b3ba93034e561910" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34741 } }
I20260810 14:42:54.650969 18886 raft_consensus.cc:348] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3376673d7694334ab808f3d47552e87" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45935 } } peers { permanent_uuid: "35fe530eeeb348aa8f1554588bda5e4f" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35485 } } peers { permanent_uuid: "b0d6e0f4e9a34456b3ba93034e561910" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34741 } }
I20260810 14:42:54.651376 18886 raft_consensus.cc:374] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:54.651376 18883 raft_consensus.cc:374] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:54.651386 18884 ts_tablet_manager.cc:1428] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260810 14:42:54.651473 18883 raft_consensus.cc:729] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3376673d7694334ab808f3d47552e87, State: Initialized, Role: FOLLOWER
I20260810 14:42:54.651475 18886 raft_consensus.cc:729] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35fe530eeeb348aa8f1554588bda5e4f, State: Initialized, Role: FOLLOWER
I20260810 14:42:54.652150 18886 consensus_queue.cc:260] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3376673d7694334ab808f3d47552e87" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45935 } } peers { permanent_uuid: "35fe530eeeb348aa8f1554588bda5e4f" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35485 } } peers { permanent_uuid: "b0d6e0f4e9a34456b3ba93034e561910" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34741 } }
I20260810 14:42:54.652155 18883 consensus_queue.cc:260] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3376673d7694334ab808f3d47552e87" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45935 } } peers { permanent_uuid: "35fe530eeeb348aa8f1554588bda5e4f" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35485 } } peers { permanent_uuid: "b0d6e0f4e9a34456b3ba93034e561910" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34741 } }
I20260810 14:42:54.653661 18863 heartbeater.cc:503] Master 127.21.55.126:42415 was elected leader, sending a full tablet report...
I20260810 14:42:54.653906 18886 ts_tablet_manager.cc:1428] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260810 14:42:54.654263 18883 ts_tablet_manager.cc:1428] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
W20260810 14:42:54.754717 18465 tablet.cc:2369] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:42:54.767027 18864 tablet.cc:2369] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:42:54.777529 18666 tablet.cc:2369] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:42:54.878561 18895 raft_consensus.cc:482] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:42:54.878924 18895 raft_consensus.cc:504] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3376673d7694334ab808f3d47552e87" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45935 } } peers { permanent_uuid: "35fe530eeeb348aa8f1554588bda5e4f" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35485 } } peers { permanent_uuid: "b0d6e0f4e9a34456b3ba93034e561910" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34741 } }
I20260810 14:42:54.880688 18895 leader_election.cc:288] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3376673d7694334ab808f3d47552e87 (127.21.55.67:45935), 35fe530eeeb348aa8f1554588bda5e4f (127.21.55.65:35485)
I20260810 14:42:54.890010 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.881041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:35485 (local address 127.21.55.66:45487)
0810 14:42:54.882099 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:54.882128 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:54.882353 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:54.882823 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:54.882828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:54.882854 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:54.883262 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.883272 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.884346 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.884351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.886986 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.887026 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.889025 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.889063 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.889276 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.889321 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 14:42:54.889439 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 14:42:54.889589 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":1,"thread_start_us":681,"threads_started":1}
I20260810 14:42:54.890064 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.881090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:45935 (local address 127.21.55.66:35571)
0810 14:42:54.882363 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:54.882380 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:54.882571 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:54.882990 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:54.883000 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:54.883090 (+ 90us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:54.883499 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.883511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.884895 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.884900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.887445 (+ 2545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:54.887467 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.889185 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.889192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:54.889395 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.889439 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 14:42:54.889522 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 14:42:54.889622 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1159,"mutex_wait_us":61,"spinlock_wait_cycles":43392,"thread_start_us":963,"threads_started":1}
I20260810 14:42:54.890386 18880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.881239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:45487 (local address 127.21.55.65:35485)
0810 14:42:54.881459 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:54.881465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:54.882227 (+ 762us) server_negotiation.cc:407] Connection header received
0810 14:42:54.882523 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:54.882575 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:54.882692 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:54.882788 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:54.883425 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.884202 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.887422 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.888754 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.888950 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.889030 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:54.889633 (+ 603us) server_negotiation.cc:299] Negotiation successful
0810 14:42:54.889839 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 14:42:54.890707 18879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.881739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:35571 (local address 127.21.55.67:45935)
0810 14:42:54.881929 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:54.881935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:54.882457 (+ 522us) server_negotiation.cc:407] Connection header received
0810 14:42:54.882720 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:54.882728 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:54.882809 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:54.882891 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:54.883684 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.884751 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.887674 (+ 2923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.888959 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.889147 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.889194 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:54.889663 (+ 469us) server_negotiation.cc:299] Negotiation successful
0810 14:42:54.889839 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 14:42:54.891791 18399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850f511a73fb48b7886372bf272a4e13" candidate_uuid: "b0d6e0f4e9a34456b3ba93034e561910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35fe530eeeb348aa8f1554588bda5e4f" is_pre_election: true
I20260810 14:42:54.892139 18789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850f511a73fb48b7886372bf272a4e13" candidate_uuid: "b0d6e0f4e9a34456b3ba93034e561910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3376673d7694334ab808f3d47552e87" is_pre_election: true
I20260810 14:42:54.892336 18399 raft_consensus.cc:2454] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0d6e0f4e9a34456b3ba93034e561910 in term 0.
I20260810 14:42:54.892850 18789 raft_consensus.cc:2454] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0d6e0f4e9a34456b3ba93034e561910 in term 0.
I20260810 14:42:54.893438 18495 leader_election.cc:302] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35fe530eeeb348aa8f1554588bda5e4f, b0d6e0f4e9a34456b3ba93034e561910; no voters:
I20260810 14:42:54.893942 18895 raft_consensus.cc:2781] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:42:54.894093 18895 raft_consensus.cc:482] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:42:54.894207 18895 raft_consensus.cc:3037] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:54.896481 18895 raft_consensus.cc:504] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3376673d7694334ab808f3d47552e87" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45935 } } peers { permanent_uuid: "35fe530eeeb348aa8f1554588bda5e4f" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35485 } } peers { permanent_uuid: "b0d6e0f4e9a34456b3ba93034e561910" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34741 } }
I20260810 14:42:54.897336 18895 leader_election.cc:288] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [CANDIDATE]: Term 1 election: Requested vote from peers d3376673d7694334ab808f3d47552e87 (127.21.55.67:45935), 35fe530eeeb348aa8f1554588bda5e4f (127.21.55.65:35485)
I20260810 14:42:54.897913 18789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850f511a73fb48b7886372bf272a4e13" candidate_uuid: "b0d6e0f4e9a34456b3ba93034e561910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3376673d7694334ab808f3d47552e87"
I20260810 14:42:54.898172 18789 raft_consensus.cc:3037] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:54.898195 18399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "850f511a73fb48b7886372bf272a4e13" candidate_uuid: "b0d6e0f4e9a34456b3ba93034e561910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35fe530eeeb348aa8f1554588bda5e4f"
I20260810 14:42:54.898434 18399 raft_consensus.cc:3037] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:54.900518 18789 raft_consensus.cc:2454] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0d6e0f4e9a34456b3ba93034e561910 in term 1.
I20260810 14:42:54.900702 18399 raft_consensus.cc:2454] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0d6e0f4e9a34456b3ba93034e561910 in term 1.
I20260810 14:42:54.901010 18497 leader_election.cc:302] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0d6e0f4e9a34456b3ba93034e561910, d3376673d7694334ab808f3d47552e87; no voters:
I20260810 14:42:54.901315 18895 raft_consensus.cc:2781] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:42:54.901769 18895 raft_consensus.cc:686] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 1 LEADER]: Becoming Leader. State: Replica: b0d6e0f4e9a34456b3ba93034e561910, State: Running, Role: LEADER
I20260810 14:42:54.902257 18895 consensus_queue.cc:237] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3376673d7694334ab808f3d47552e87" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45935 } } peers { permanent_uuid: "35fe530eeeb348aa8f1554588bda5e4f" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35485 } } peers { permanent_uuid: "b0d6e0f4e9a34456b3ba93034e561910" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34741 } }
I20260810 14:42:54.907279 18184 catalog_manager.cc:5495] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0d6e0f4e9a34456b3ba93034e561910 (127.21.55.66). New cstate: current_term: 1 leader_uuid: "b0d6e0f4e9a34456b3ba93034e561910" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3376673d7694334ab808f3d47552e87" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35fe530eeeb348aa8f1554588bda5e4f" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0d6e0f4e9a34456b3ba93034e561910" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 34741 } health_report { overall_health: HEALTHY } } }
I20260810 14:42:54.958190 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:54.952518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38164 (local address 127.21.55.66:34741)
0810 14:42:54.952815 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:54.952820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:54.952838 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:42:54.952932 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:54.952936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:54.953050 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:54.953216 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:42:54.953578 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.954385 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.956006 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:54.956427 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:54.956573 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:54.957066 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:42:54.957581 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:42:54.957742 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:54.957825 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 14:42:54.957903 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260810 14:42:54.968822 18399 raft_consensus.cc:1261] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f [term 1 FOLLOWER]: Refusing update from remote peer b0d6e0f4e9a34456b3ba93034e561910: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:42:54.968837 18789 raft_consensus.cc:1261] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87 [term 1 FOLLOWER]: Refusing update from remote peer b0d6e0f4e9a34456b3ba93034e561910: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:42:54.970088 18895 consensus_queue.cc:1035] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3376673d7694334ab808f3d47552e87" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:42:54.970451 18901 consensus_queue.cc:1035] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35fe530eeeb348aa8f1554588bda5e4f" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:42:55.004066 18185 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:42:55.004293 18185 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:42:55.007623 18185 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8b4c44f97d74b36bf9f4432cf74f20d: Sending DeleteTablet for 3 replicas of tablet 850f511a73fb48b7886372bf272a4e13
I20260810 14:42:55.008808 18366 tablet_service.cc:1514] Processing DeleteTablet for tablet 850f511a73fb48b7886372bf272a4e13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:55 UTC) from {username='slave'} at 127.0.0.1:43480
I20260810 14:42:55.008810 18764 tablet_service.cc:1514] Processing DeleteTablet for tablet 850f511a73fb48b7886372bf272a4e13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:55 UTC) from {username='slave'} at 127.0.0.1:45074
I20260810 14:42:55.009572 18563 tablet_service.cc:1514] Processing DeleteTablet for tablet 850f511a73fb48b7886372bf272a4e13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:55 UTC) from {username='slave'} at 127.0.0.1:38148
I20260810 14:42:55.009694 18926 tablet_replica.cc:330] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: stopping tablet replica
I20260810 14:42:55.010223 18926 raft_consensus.cc:2229] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:42:55.010792 18926 raft_consensus.cc:2258] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:42:55.011801 18924 tablet_replica.cc:330] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: stopping tablet replica
I20260810 14:42:55.011871 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 18271
I20260810 14:42:55.012491 18924 raft_consensus.cc:2229] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:42:55.013008 18926 ts_tablet_manager.cc:1905] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:55.013015 18924 raft_consensus.cc:2258] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:42:55.015138 18924 ts_tablet_manager.cc:1905] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:55.022347 18924 ts_tablet_manager.cc:1918] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:42:55.022671 18924 log.cc:1196] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/850f511a73fb48b7886372bf272a4e13
I20260810 14:42:55.023336 18924 ts_tablet_manager.cc:1939] T 850f511a73fb48b7886372bf272a4e13 P d3376673d7694334ab808f3d47552e87: Deleting consensus metadata
I20260810 14:42:55.024204 18927 tablet_replica.cc:330] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: stopping tablet replica
I20260810 14:42:55.024556 18927 raft_consensus.cc:2229] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:42:55.025233 18927 raft_consensus.cc:2258] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:42:55.025348 18166 catalog_manager.cc:4841] TS d3376673d7694334ab808f3d47552e87 (127.21.55.67:45935): tablet 850f511a73fb48b7886372bf272a4e13 (table test-table [id=435590c7ad2b4a67a94a1a79118dc2c9]) successfully deleted
I20260810 14:42:55.026471 18926 ts_tablet_manager.cc:1918] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:42:55.026678 18926 log.cc:1196] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/850f511a73fb48b7886372bf272a4e13
I20260810 14:42:55.027261 18926 ts_tablet_manager.cc:1939] T 850f511a73fb48b7886372bf272a4e13 P 35fe530eeeb348aa8f1554588bda5e4f: Deleting consensus metadata
I20260810 14:42:55.027639 18927 ts_tablet_manager.cc:1905] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:55.029081 18162 catalog_manager.cc:4841] TS 35fe530eeeb348aa8f1554588bda5e4f (127.21.55.65:35485): tablet 850f511a73fb48b7886372bf272a4e13 (table test-table [id=435590c7ad2b4a67a94a1a79118dc2c9]) successfully deleted
I20260810 14:42:55.035475 18880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:55.012867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.21.55.65:35485)
0810 14:42:55.013048 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:55.013057 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:55.026061 (+ 13004us) server_negotiation.cc:407] Connection header received
0810 14:42:55.027059 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:55.027065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:55.027172 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:55.027272 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:55.028134 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:55.029229 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:55.031350 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:55.032040 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:55.032157 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:55.033692 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:55.033704 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:55.033720 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:42:55.034482 (+ 762us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:55.034595 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:42:55.034622 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:55.034707 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:55.034967 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 14:42:55.035098 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:42:55.035439 18927 ts_tablet_manager.cc:1918] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:42:55.035619 18927 log.cc:1196] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/850f511a73fb48b7886372bf272a4e13
I20260810 14:42:55.036248 18927 ts_tablet_manager.cc:1939] T 850f511a73fb48b7886372bf272a4e13 P b0d6e0f4e9a34456b3ba93034e561910: Deleting consensus metadata
I20260810 14:42:55.037902 18165 catalog_manager.cc:4841] TS b0d6e0f4e9a34456b3ba93034e561910 (127.21.55.66:34741): tablet 850f511a73fb48b7886372bf272a4e13 (table test-table [id=435590c7ad2b4a67a94a1a79118dc2c9]) successfully deleted
I20260810 14:42:55.038913 18458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:55.139717 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 18271
I20260810 14:42:55.172449 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 18470
I20260810 14:42:55.181115 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:55.173317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38180 (local address 127.21.55.66:34741)
0810 14:42:55.173500 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:55.173504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:55.173561 (+ 57us) server_negotiation.cc:407] Connection header received
0810 14:42:55.173805 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:55.173817 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:55.173907 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:55.174018 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:55.174522 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:55.175666 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:55.177799 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:55.178271 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:55.178382 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:55.179626 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:55.179634 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:55.179648 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:42:55.180303 (+ 655us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:55.180355 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:42:55.180374 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:55.180483 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:55.180653 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 14:42:55.180730 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:42:55.181902 18656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:55.305238 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 18470
I20260810 14:42:55.331705 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 18672
I20260810 14:42:55.340269 18879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:55.332588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45088 (local address 127.21.55.67:45935)
0810 14:42:55.332849 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:55.332857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:55.332961 (+ 104us) server_negotiation.cc:407] Connection header received
0810 14:42:55.333170 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:55.333182 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:55.333264 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:55.333388 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:55.333798 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:55.334516 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:55.335952 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:55.336362 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:55.336496 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:55.338272 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:55.338286 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:55.338308 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:42:55.339196 (+ 888us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:55.339265 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:42:55.339291 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:55.339465 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:55.339637 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 14:42:55.339793 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260810 14:42:55.341337 18856 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:55.436275 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 18672
I20260810 14:42:55.463419 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 18141
I20260810 14:42:55.464906 18227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:42:55.585839 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 18141
2026-08-10T14:42:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3287 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:42:55 dist-test-slave-h6n0 krb5kdc[18937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:42:55 dist-test-slave-h6n0 krb5kdc[18937](info): set up 2 sockets
Aug 10 14:42:55 dist-test-slave-h6n0 krb5kdc[18937](info): commencing operation
krb5kdc: starting...
W20260810 14:42:57.001340 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.363s user 0.011s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:42:57 dist-test-slave-h6n0 krb5kdc[18937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:42:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:42:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:42:57.302024 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:46307
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:46307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:42:57.652729 18955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:57.653000 18955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:57.653097 18955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:57.660007 18955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:42:57.660117 18955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:57.660164 18955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:57.660218 18955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:42:57.660262 18955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:42:57.669756 18955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:46307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:46307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:57.670943 18955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:57.672163 18955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:57.684067 18963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:57.684367 18964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:57.684309 18967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:57.686035 18955 server_base.cc:1034] running on GCE node
I20260810 14:42:57.687278 18955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:57.689106 18955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:57.690429 18955 hybrid_clock.cc:648] HybridClock initialized: now 1786372977690367 us; error 45 us; skew 500 ppm
I20260810 14:42:57.690825 18955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:57 dist-test-slave-h6n0 krb5kdc[18937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372977, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:57.693991 18955 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:42:57.695472 18955 webserver.cc:460] Webserver started at http://127.21.55.126:35683/ using document root <none> and password file <none>
I20260810 14:42:57.696156 18955 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:57.696306 18955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:57.696621 18955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:57.699580 18955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "ff228e9ccee843c3b285952cf92371df"
format_stamp: "Formatted at 2026-08-10 14:42:57 on dist-test-slave-h6n0"
server_key: "b3e9273d0a12ddd5dcb18427a05028a3"
server_key_iv: "37e692a0af5cfcd52a9738dcbb80cbc1"
server_key_version: "encryptionkey@0"
I20260810 14:42:57.700333 18955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "ff228e9ccee843c3b285952cf92371df"
format_stamp: "Formatted at 2026-08-10 14:42:57 on dist-test-slave-h6n0"
server_key: "b3e9273d0a12ddd5dcb18427a05028a3"
server_key_iv: "37e692a0af5cfcd52a9738dcbb80cbc1"
server_key_version: "encryptionkey@0"
I20260810 14:42:57.705727 18955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:42:57.710479 18974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:57.712035 18955 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:42:57.712249 18955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "ff228e9ccee843c3b285952cf92371df"
format_stamp: "Formatted at 2026-08-10 14:42:57 on dist-test-slave-h6n0"
server_key: "b3e9273d0a12ddd5dcb18427a05028a3"
server_key_iv: "37e692a0af5cfcd52a9738dcbb80cbc1"
server_key_version: "encryptionkey@0"
I20260810 14:42:57.712468 18955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:57.729821 18955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:57.734182 18955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:57.734483 18955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:57.755558 18955 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:46307
I20260810 14:42:57.755612 19049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:46307 every 8 connection(s)
I20260810 14:42:57.756912 18955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:42:57.760769 19050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:57.762207 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 18955
I20260810 14:42:57.762413 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:42:57.763063 21725 external_mini_cluster.cc:1428] Setting key 99c30d172038f7fff69bae0d8a7a0289
I20260810 14:42:57.775465 19050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df: Bootstrap starting.
I20260810 14:42:57.778687 19050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df: Neither blocks nor log segments found. Creating new log.
Aug 10 14:42:57 dist-test-slave-h6n0 krb5kdc[18937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:42:57.780453 19050 log.cc:824] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:57.784338 19050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df: No bootstrap required, opened a new log
I20260810 14:42:57.789909 19053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:57.765013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38610 (local address 127.21.55.126:46307)
0810 14:42:57.766316 (+ 1303us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:57.766333 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:57.766439 (+ 106us) server_negotiation.cc:407] Connection header received
0810 14:42:57.768418 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:57.768447 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:57.768831 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:57.769257 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:57.770936 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:57.773028 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:57.774429 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:57.775494 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:57.775677 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:57.780770 (+ 5093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:57.780777 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:57.780791 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:57.780849 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:57.785320 (+ 4471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:57.786391 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:57.786398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:57.786403 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:57.786700 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:57.787169 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:57.787173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:57.787175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:57.787728 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:57.787967 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:57.788469 (+ 502us) server_negotiation.cc:299] Negotiation successful
0810 14:42:57.788851 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":988,"thread_start_us":663,"threads_started":1}
I20260810 14:42:57.793908 19050 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff228e9ccee843c3b285952cf92371df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46307 } }
I20260810 14:42:57.794238 19050 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:57.794376 19050 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff228e9ccee843c3b285952cf92371df, State: Initialized, Role: FOLLOWER
I20260810 14:42:57.795114 19050 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff228e9ccee843c3b285952cf92371df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46307 } }
I20260810 14:42:57.795393 19050 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:42:57.795568 19050 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:42:57.795760 19050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:57.797899 19050 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff228e9ccee843c3b285952cf92371df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46307 } }
I20260810 14:42:57.798396 19050 leader_election.cc:302] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff228e9ccee843c3b285952cf92371df; no voters:
I20260810 14:42:57.799011 19050 leader_election.cc:288] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:42:57.799152 19055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:42:57.799834 19055 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [term 1 LEADER]: Becoming Leader. State: Replica: ff228e9ccee843c3b285952cf92371df, State: Running, Role: LEADER
I20260810 14:42:57.800386 19055 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff228e9ccee843c3b285952cf92371df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46307 } }
I20260810 14:42:57.801486 19050 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:42:57.803418 19057 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff228e9ccee843c3b285952cf92371df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff228e9ccee843c3b285952cf92371df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46307 } } }
I20260810 14:42:57.803432 19058 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff228e9ccee843c3b285952cf92371df. Latest consensus state: current_term: 1 leader_uuid: "ff228e9ccee843c3b285952cf92371df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff228e9ccee843c3b285952cf92371df" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46307 } } }
I20260810 14:42:57.803988 19057 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [sys.catalog]: This master's current role is: LEADER
I20260810 14:42:57.804018 19058 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df [sys.catalog]: This master's current role is: LEADER
I20260810 14:42:57.805514 19064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:42:57.813494 19064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:42:57.824817 19064 catalog_manager.cc:1347] Generated new cluster ID: 2e1de13e16d14858ab59b40eaf950a38
I20260810 14:42:57.824939 19064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:42:57.842391 19064 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:42:57.844925 19064 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:42:57.860682 19064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df: Generated new TSK 0
I20260810 14:42:57.861819 19064 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:42:57.949810 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46307
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:42:58.333849 19081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:58.334227 19081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:58.334409 19081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:58.343618 19081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:58.344041 19081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:58.344306 19081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:42:58.355746 19081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:58.357071 19081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:58.358350 19081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:58.373572 19089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:58.373528 19092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:58.373622 19088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:58.376348 19081 server_base.cc:1034] running on GCE node
I20260810 14:42:58.377046 19081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:58.378111 19081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:58.379544 19081 hybrid_clock.cc:648] HybridClock initialized: now 1786372978379436 us; error 103 us; skew 500 ppm
I20260810 14:42:58.379968 19081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:58 dist-test-slave-h6n0 krb5kdc[18937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372978, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:58.383477 19081 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:42:58.385176 19081 webserver.cc:460] Webserver started at http://127.21.55.65:45177/ using document root <none> and password file <none>
I20260810 14:42:58.385867 19081 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:58.385979 19081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:58.386240 19081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:58.389159 19081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "b258f2c1a4354048afbf073712ecd635"
format_stamp: "Formatted at 2026-08-10 14:42:58 on dist-test-slave-h6n0"
server_key: "18104f4315237483660101914820e4bd"
server_key_iv: "be2fc94b28f748172cb9c120473b3542"
server_key_version: "encryptionkey@0"
I20260810 14:42:58.389886 19081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "b258f2c1a4354048afbf073712ecd635"
format_stamp: "Formatted at 2026-08-10 14:42:58 on dist-test-slave-h6n0"
server_key: "18104f4315237483660101914820e4bd"
server_key_iv: "be2fc94b28f748172cb9c120473b3542"
server_key_version: "encryptionkey@0"
I20260810 14:42:58.395713 19081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:42:58.400144 19099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:58.403024 19081 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:42:58.403411 19081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "b258f2c1a4354048afbf073712ecd635"
format_stamp: "Formatted at 2026-08-10 14:42:58 on dist-test-slave-h6n0"
server_key: "18104f4315237483660101914820e4bd"
server_key_iv: "be2fc94b28f748172cb9c120473b3542"
server_key_version: "encryptionkey@0"
I20260810 14:42:58.403753 19081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:58.417413 19081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:58.422242 19081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:58.422634 19081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:58.423707 19081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:58.425287 19081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:58.425398 19081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:58.425562 19081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:58.425640 19081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:58.468235 19081 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:33059
I20260810 14:42:58.468371 19257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:33059 every 8 connection(s)
I20260810 14:42:58.469770 19081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:42:58.477234 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 19081
I20260810 14:42:58.477471 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:42:58.478119 21725 external_mini_cluster.cc:1428] Setting key 323a65693f095ea94c2b2bbb620ace97
Aug 10 14:42:58 dist-test-slave-h6n0 krb5kdc[18937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372978, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:42:58.495290 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:58.473414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:46497 (local address 127.21.55.126:46307)
0810 14:42:58.474010 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:58.474018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:58.475462 (+ 1444us) server_negotiation.cc:407] Connection header received
0810 14:42:58.477378 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:58.477384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:58.477492 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:58.477619 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:58.479509 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:58.480549 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:58.482173 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:58.482649 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:58.482786 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:58.487507 (+ 4721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:58.487512 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:58.487515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:58.487545 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:58.491819 (+ 4274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:58.492630 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:58.492639 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:58.492645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:58.492805 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:58.493275 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:58.493282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:58.493288 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:58.493677 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:58.493832 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:58.494673 (+ 841us) server_negotiation.cc:299] Negotiation successful
0810 14:42:58.494951 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":283,"threads_started":1}
I20260810 14:42:58.496536 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:58.474125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46307 (local address 127.21.55.65:46497)
0810 14:42:58.475255 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:58.475310 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:58.477058 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:58.477897 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:58.477905 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:58.478494 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:58.479266 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:58.479285 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:58.480722 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:58.480733 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:58.482000 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:58.482017 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:58.482900 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:58.482908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:58.483149 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:58.484030 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:42:58.484054 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:58.487261 (+ 3207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:58.492003 (+ 4742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:58.492011 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:58.492025 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:58.492475 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:58.492940 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:58.492943 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:58.492945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:58.493148 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:58.493895 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:58.493902 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:58.494234 (+ 332us) client_negotiation.cc:769] Sending connection context
0810 14:42:58.494468 (+ 234us) client_negotiation.cc:240] Negotiation successful
0810 14:42:58.494948 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"thread_start_us":445,"threads_started":1}
I20260810 14:42:58.498191 19258 heartbeater.cc:347] Connected to a master server at 127.21.55.126:46307
I20260810 14:42:58.498643 19258 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:58.499779 19258 heartbeater.cc:511] Master 127.21.55.126:46307 requested a full tablet report, sending...
I20260810 14:42:58.502784 18995 ts_manager.cc:194] Registered new tserver with Master: b258f2c1a4354048afbf073712ecd635 (127.21.55.65:33059)
I20260810 14:42:58.504380 18995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:46497
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:42:58.551088 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46307
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:42:58.904747 19269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:58.905011 19269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:58.905167 19269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:58.913115 19269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:58.913219 19269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:58.913426 19269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:42:58.923245 19269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:58.924510 19269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:58.925786 19269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:42:58.938971 19274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:58.939035 19278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:58.938946 19275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:58.940160 19269 server_base.cc:1034] running on GCE node
I20260810 14:42:58.940802 19269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:58.941782 19269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:58.943027 19269 hybrid_clock.cc:648] HybridClock initialized: now 1786372978942980 us; error 68 us; skew 500 ppm
I20260810 14:42:58.943360 19269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:58 dist-test-slave-h6n0 krb5kdc[18937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372978, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:58.947002 19269 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:42:58.948606 19269 webserver.cc:460] Webserver started at http://127.21.55.66:35223/ using document root <none> and password file <none>
I20260810 14:42:58.949311 19269 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:58.949465 19269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:58.950031 19269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:58.953927 19269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "e4f31a4b99b844ddba2c9cc108b2e441"
format_stamp: "Formatted at 2026-08-10 14:42:58 on dist-test-slave-h6n0"
server_key: "c8df14e8cc518a48b086fc1f2bad1285"
server_key_iv: "cf1df6138584cdbdbf1336c35001e9ac"
server_key_version: "encryptionkey@0"
I20260810 14:42:58.954813 19269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "e4f31a4b99b844ddba2c9cc108b2e441"
format_stamp: "Formatted at 2026-08-10 14:42:58 on dist-test-slave-h6n0"
server_key: "c8df14e8cc518a48b086fc1f2bad1285"
server_key_iv: "cf1df6138584cdbdbf1336c35001e9ac"
server_key_version: "encryptionkey@0"
I20260810 14:42:58.960005 19269 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 14:42:58.963868 19286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:58.965549 19269 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:42:58.965795 19269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "e4f31a4b99b844ddba2c9cc108b2e441"
format_stamp: "Formatted at 2026-08-10 14:42:58 on dist-test-slave-h6n0"
server_key: "c8df14e8cc518a48b086fc1f2bad1285"
server_key_iv: "cf1df6138584cdbdbf1336c35001e9ac"
server_key_version: "encryptionkey@0"
I20260810 14:42:58.966007 19269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:58.996075 19269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:59.000871 19269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:59.001199 19269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:59.002595 19269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:59.004271 19269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:59.004381 19269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:59.004539 19269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:59.004616 19269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:59.047649 19269 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:38429
I20260810 14:42:59.047698 19455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:38429 every 8 connection(s)
I20260810 14:42:59.049209 19269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:42:59.054970 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 19269
I20260810 14:42:59.055181 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:42:59.055632 21725 external_mini_cluster.cc:1428] Setting key e2f53ec2e67ba0629aacd635018738af
Aug 10 14:42:59 dist-test-slave-h6n0 krb5kdc[18937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372978, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:42:59.072954 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.052839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:33267 (local address 127.21.55.126:46307)
0810 14:42:59.053528 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:59.053534 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:59.054709 (+ 1175us) server_negotiation.cc:407] Connection header received
0810 14:42:59.056943 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:59.056948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:59.057041 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:59.057162 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:59.058860 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.059582 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.061100 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.061533 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.061667 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.066198 (+ 4531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:59.066214 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:59.066219 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:59.066257 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:59.069293 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:59.070097 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:59.070102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:59.070106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:59.070305 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:59.070849 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:59.070864 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:59.070868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:59.071173 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:59.071333 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:59.072055 (+ 722us) server_negotiation.cc:299] Negotiation successful
0810 14:42:59.072286 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":400,"threads_started":1}
I20260810 14:42:59.074251 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.053688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46307 (local address 127.21.55.66:33267)
0810 14:42:59.054456 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:59.054506 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:59.056679 (+ 2173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:59.057429 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:59.057440 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:59.058060 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:59.058690 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.058708 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.059729 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.059734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.060946 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.060958 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.061759 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.061767 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.062000 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.062756 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:42:59.062777 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:59.065732 (+ 2955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:59.069461 (+ 3729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:59.069471 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:59.069487 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:59.069961 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:59.070423 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:59.070428 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:59.070431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:59.070721 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:59.071310 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:59.071318 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:59.071656 (+ 338us) client_negotiation.cc:769] Sending connection context
0810 14:42:59.071930 (+ 274us) client_negotiation.cc:240] Negotiation successful
0810 14:42:59.072700 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":309,"threads_started":1}
I20260810 14:42:59.075960 19456 heartbeater.cc:347] Connected to a master server at 127.21.55.126:46307
I20260810 14:42:59.076526 19456 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:59.077559 19456 heartbeater.cc:511] Master 127.21.55.126:46307 requested a full tablet report, sending...
I20260810 14:42:59.079666 18995 ts_manager.cc:194] Registered new tserver with Master: e4f31a4b99b844ddba2c9cc108b2e441 (127.21.55.66:38429)
I20260810 14:42:59.080999 18995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:33267
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:42:59.126829 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46307
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:42:59.480883 19467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:42:59.481138 19467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:42:59.481295 19467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:42:59.488597 19467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:42:59.488696 19467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:42:59.488899 19467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:42:59.498795 19467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:42:59.499958 19467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:42:59.501220 19467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:42:59.507467 19258 heartbeater.cc:503] Master 127.21.55.126:46307 was elected leader, sending a full tablet report...
W20260810 14:42:59.514503 19474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:59.514794 19473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:42:59.514813 19477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:42:59.515431 19467 server_base.cc:1034] running on GCE node
I20260810 14:42:59.516054 19467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:42:59.517055 19467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:42:59.518260 19467 hybrid_clock.cc:648] HybridClock initialized: now 1786372979518218 us; error 36 us; skew 500 ppm
I20260810 14:42:59.518632 19467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:42:59 dist-test-slave-h6n0 krb5kdc[18937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372979, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:42:59.526023 19467 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:42:59.527578 19467 webserver.cc:460] Webserver started at http://127.21.55.67:43937/ using document root <none> and password file <none>
I20260810 14:42:59.528287 19467 fs_manager.cc:359] Metadata directory not provided
I20260810 14:42:59.528434 19467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:42:59.528755 19467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:42:59.531646 19467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "92e1a7c0c6f14ea299c80d9167cccab3"
format_stamp: "Formatted at 2026-08-10 14:42:59 on dist-test-slave-h6n0"
server_key: "2304a0bfd89b083f59331eef6becfc73"
server_key_iv: "d3816b1ee56e67bd3228413b01cbd54f"
server_key_version: "encryptionkey@0"
I20260810 14:42:59.532423 19467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "92e1a7c0c6f14ea299c80d9167cccab3"
format_stamp: "Formatted at 2026-08-10 14:42:59 on dist-test-slave-h6n0"
server_key: "2304a0bfd89b083f59331eef6becfc73"
server_key_iv: "d3816b1ee56e67bd3228413b01cbd54f"
server_key_version: "encryptionkey@0"
I20260810 14:42:59.537556 19467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 14:42:59.540926 19487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:59.542109 19467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:42:59.542281 19467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "92e1a7c0c6f14ea299c80d9167cccab3"
format_stamp: "Formatted at 2026-08-10 14:42:59 on dist-test-slave-h6n0"
server_key: "2304a0bfd89b083f59331eef6becfc73"
server_key_iv: "d3816b1ee56e67bd3228413b01cbd54f"
server_key_version: "encryptionkey@0"
I20260810 14:42:59.542475 19467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:42:59.564986 19467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:42:59.569729 19467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:42:59.570042 19467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:42:59.571395 19467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:42:59.572818 19467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:42:59.572930 19467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:59.573083 19467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:42:59.573158 19467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:42:59.610883 19467 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:43893
I20260810 14:42:59.611006 19658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:43893 every 8 connection(s)
I20260810 14:42:59.612303 19467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:42:59.620721 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 19467
I20260810 14:42:59.620904 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:42:59.621378 21725 external_mini_cluster.cc:1428] Setting key 092e8a95f2b12215731934c541c6d659
Aug 10 14:42:59 dist-test-slave-h6n0 krb5kdc[18937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372979, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:42:59.635070 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.616398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:53719 (local address 127.21.55.126:46307)
0810 14:42:59.616942 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:59.616947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:59.617889 (+ 942us) server_negotiation.cc:407] Connection header received
0810 14:42:59.619814 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:59.619819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:59.619932 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:59.620072 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:59.621738 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.622632 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.625210 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.625700 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.625832 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.629325 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:59.629330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:59.629334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:59.629369 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:59.632127 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:59.632873 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:59.632878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:59.632880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:59.633019 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:59.633454 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:59.633458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:59.633460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:59.633718 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:59.633841 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:59.634578 (+ 737us) server_negotiation.cc:299] Negotiation successful
0810 14:42:59.634760 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":265,"threads_started":1}
I20260810 14:42:59.636088 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.616951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46307 (local address 127.21.55.67:53719)
0810 14:42:59.617684 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:59.617728 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:59.619547 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:59.620316 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:59.620324 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:59.620910 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:42:59.621554 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.621573 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.622964 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.622970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.624985 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.625001 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.625950 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.625955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.626169 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.626883 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:42:59.626905 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:42:59.629126 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:42:59.632262 (+ 3136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:59.632268 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:59.632278 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:59.632710 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:59.633121 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:42:59.633124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:42:59.633127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:42:59.633350 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:42:59.633855 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:42:59.633861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:42:59.634191 (+ 330us) client_negotiation.cc:769] Sending connection context
0810 14:42:59.634427 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 14:42:59.634896 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":294,"threads_started":1}
I20260810 14:42:59.637610 19661 heartbeater.cc:347] Connected to a master server at 127.21.55.126:46307
I20260810 14:42:59.637926 19661 heartbeater.cc:464] Registering TS with master...
I20260810 14:42:59.638695 19661 heartbeater.cc:511] Master 127.21.55.126:46307 requested a full tablet report, sending...
I20260810 14:42:59.640022 18995 ts_manager.cc:194] Registered new tserver with Master: 92e1a7c0c6f14ea299c80d9167cccab3 (127.21.55.67:43893)
I20260810 14:42:59.640817 18995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:53719
I20260810 14:42:59.650013 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:42:59.668198 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.654297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56034 (local address 127.21.55.126:46307)
0810 14:42:59.654491 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:59.654499 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:59.654926 (+ 427us) server_negotiation.cc:407] Connection header received
0810 14:42:59.655117 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:59.655122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:59.655197 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:59.655303 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:59.656769 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.657482 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.658874 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.659279 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.659422 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.662318 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:59.662324 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:59.662329 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:59.662371 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:59.665268 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:59.666045 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:59.666051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:59.666054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:59.666180 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:59.666708 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:59.666713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:59.666717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:59.667023 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:59.667321 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:59.667705 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 14:42:59.667890 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 14:42:59.673517 18995 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:42:59.680390 18995 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:42:59.703280 19688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.692720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:38429 (local address 127.0.0.1:57104)
0810 14:42:59.693957 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:59.694015 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:59.694213 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:59.694985 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:59.694995 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:59.695026 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:59.695483 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.695496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.697552 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.697557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.700048 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.700064 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.702226 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.702233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.702480 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.702575 (+ 95us) client_negotiation.cc:769] Sending connection context
0810 14:42:59.702702 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 14:42:59.702907 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"thread_start_us":1897,"threads_started":1}
I20260810 14:42:59.703438 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.693603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53454 (local address 127.21.55.67:43893)
0810 14:42:59.694332 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:59.694342 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:59.694959 (+ 617us) server_negotiation.cc:407] Connection header received
0810 14:42:59.695267 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:59.695274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:59.695483 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:59.695641 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:59.696604 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.698138 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.700655 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.702029 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.702131 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.702348 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:59.702939 (+ 591us) server_negotiation.cc:299] Negotiation successful
0810 14:42:59.703117 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":790,"threads_started":1}
I20260810 14:42:59.704233 19695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.693081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:43893 (local address 127.0.0.1:53454)
0810 14:42:59.694831 (+ 1750us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:59.694845 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:59.695066 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:59.695990 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:59.695994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:59.696017 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:59.696411 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.696422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.698318 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.698323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.700482 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.700496 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.702556 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.702562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.702744 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.702771 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 14:42:59.703662 (+ 891us) client_negotiation.cc:240] Negotiation successful
0810 14:42:59.703800 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1664,"spinlock_wait_cycles":896,"thread_start_us":786,"threads_started":1}
I20260810 14:42:59.704350 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.692961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57104 (local address 127.21.55.66:38429)
0810 14:42:59.693640 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:59.693651 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:59.694147 (+ 496us) server_negotiation.cc:407] Connection header received
0810 14:42:59.694442 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:59.694452 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:59.694756 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:59.694946 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:59.695708 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.697372 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.700233 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.701736 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.701879 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.702081 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:59.703800 (+ 1719us) server_negotiation.cc:299] Negotiation successful
0810 14:42:59.703975 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":355,"threads_started":1}
I20260810 14:42:59.708397 19347 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd35f7d85ad4cb887e9efe75f4dbdba (DEFAULT_TABLE table=test-table [id=cd516f2397c5409695861107d35f1019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:59.708495 19561 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd35f7d85ad4cb887e9efe75f4dbdba (DEFAULT_TABLE table=test-table [id=cd516f2397c5409695861107d35f1019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:59.709729 19347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd35f7d85ad4cb887e9efe75f4dbdba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:59.710007 19561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd35f7d85ad4cb887e9efe75f4dbdba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:59.713598 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.695524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40590 (local address 127.21.55.65:33059)
0810 14:42:59.704961 (+ 9437us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:59.704973 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:59.705004 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:42:59.705108 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:59.705116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:59.705345 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:59.705547 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:59.706152 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.707808 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.711058 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.712504 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.712656 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.712882 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:59.713119 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 14:42:59.713279 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9237,"thread_start_us":9034,"threads_started":1}
I20260810 14:42:59.714279 19688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.694975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:33059 (local address 127.0.0.1:40590)
0810 14:42:59.703528 (+ 8553us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:59.703547 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:59.703649 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:59.705770 (+ 2121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:59.705776 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:59.705801 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:59.705975 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.705986 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.708898 (+ 2912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.708911 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.710843 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.710859 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.712691 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.712698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.712910 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.712943 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:42:59.713043 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 14:42:59.714007 (+ 964us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8462,"thread_start_us":9993,"threads_started":1}
I20260810 14:42:59.720896 19163 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd35f7d85ad4cb887e9efe75f4dbdba (DEFAULT_TABLE table=test-table [id=cd516f2397c5409695861107d35f1019]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:42:59.722800 19699 tablet_bootstrap.cc:492] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: Bootstrap starting.
I20260810 14:42:59.723047 19163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd35f7d85ad4cb887e9efe75f4dbdba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:42:59.726509 19700 tablet_bootstrap.cc:492] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: Bootstrap starting.
I20260810 14:42:59.727030 19699 tablet_bootstrap.cc:654] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:59.728425 19699 log.cc:824] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:59.730262 19700 tablet_bootstrap.cc:654] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:59.731401 19700 log.cc:824] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:59.732774 19699 tablet_bootstrap.cc:492] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: No bootstrap required, opened a new log
I20260810 14:42:59.733062 19699 ts_tablet_manager.cc:1397] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260810 14:42:59.734364 19700 tablet_bootstrap.cc:492] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: No bootstrap required, opened a new log
I20260810 14:42:59.734660 19700 ts_tablet_manager.cc:1397] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260810 14:42:59.739229 19705 tablet_bootstrap.cc:492] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635: Bootstrap starting.
I20260810 14:42:59.743702 19705 tablet_bootstrap.cc:654] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635: Neither blocks nor log segments found. Creating new log.
I20260810 14:42:59.743891 19699 raft_consensus.cc:348] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38429 } } peers { permanent_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43893 } } peers { permanent_uuid: "b258f2c1a4354048afbf073712ecd635" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33059 } }
I20260810 14:42:59.744341 19699 raft_consensus.cc:374] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:59.744436 19699 raft_consensus.cc:729] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f31a4b99b844ddba2c9cc108b2e441, State: Initialized, Role: FOLLOWER
I20260810 14:42:59.744874 19705 log.cc:824] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635: Log is configured to *not* fsync() on all Append() calls
I20260810 14:42:59.745232 19699 consensus_queue.cc:260] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38429 } } peers { permanent_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43893 } } peers { permanent_uuid: "b258f2c1a4354048afbf073712ecd635" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33059 } }
I20260810 14:42:59.745532 19700 raft_consensus.cc:348] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38429 } } peers { permanent_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43893 } } peers { permanent_uuid: "b258f2c1a4354048afbf073712ecd635" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33059 } }
I20260810 14:42:59.745975 19700 raft_consensus.cc:374] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:59.746425 19700 raft_consensus.cc:729] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e1a7c0c6f14ea299c80d9167cccab3, State: Initialized, Role: FOLLOWER
I20260810 14:42:59.746585 19456 heartbeater.cc:503] Master 127.21.55.126:46307 was elected leader, sending a full tablet report...
I20260810 14:42:59.747067 19700 consensus_queue.cc:260] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38429 } } peers { permanent_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43893 } } peers { permanent_uuid: "b258f2c1a4354048afbf073712ecd635" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33059 } }
I20260810 14:42:59.747617 19699 ts_tablet_manager.cc:1428] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260810 14:42:59.748205 19661 heartbeater.cc:503] Master 127.21.55.126:46307 was elected leader, sending a full tablet report...
I20260810 14:42:59.748771 19700 ts_tablet_manager.cc:1428] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260810 14:42:59.749550 19705 tablet_bootstrap.cc:492] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635: No bootstrap required, opened a new log
I20260810 14:42:59.749841 19705 ts_tablet_manager.cc:1397] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260810 14:42:59.759577 19705 raft_consensus.cc:348] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38429 } } peers { permanent_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43893 } } peers { permanent_uuid: "b258f2c1a4354048afbf073712ecd635" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33059 } }
I20260810 14:42:59.760015 19705 raft_consensus.cc:374] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:42:59.760114 19705 raft_consensus.cc:729] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b258f2c1a4354048afbf073712ecd635, State: Initialized, Role: FOLLOWER
I20260810 14:42:59.760672 19705 consensus_queue.cc:260] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38429 } } peers { permanent_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43893 } } peers { permanent_uuid: "b258f2c1a4354048afbf073712ecd635" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33059 } }
I20260810 14:42:59.762349 19705 ts_tablet_manager.cc:1428] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635: Time spent starting tablet: real 0.012s user 0.009s sys 0.005s
W20260810 14:42:59.801841 19457 tablet.cc:2369] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:42:59.865767 19662 tablet.cc:2369] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:42:59.870545 19714 raft_consensus.cc:482] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:42:59.870931 19714 raft_consensus.cc:504] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38429 } } peers { permanent_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43893 } } peers { permanent_uuid: "b258f2c1a4354048afbf073712ecd635" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33059 } }
I20260810 14:42:59.872674 19714 leader_election.cc:288] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4f31a4b99b844ddba2c9cc108b2e441 (127.21.55.66:38429), 92e1a7c0c6f14ea299c80d9167cccab3 (127.21.55.67:43893)
I20260810 14:42:59.879309 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.873217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:40083 (local address 127.21.55.66:38429)
0810 14:42:59.873423 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:59.873427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:59.874149 (+ 722us) server_negotiation.cc:407] Connection header received
0810 14:42:59.874397 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:59.874411 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:59.874470 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:59.874604 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:59.875146 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.875850 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.877593 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.878354 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.878568 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.878645 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:59.878916 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 14:42:59.879039 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:42:59.879482 19715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.873271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:38429 (local address 127.21.55.65:40083)
0810 14:42:59.874030 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:59.874046 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:59.874238 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:59.874631 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:59.874635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:59.874661 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:59.874997 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.875005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.875991 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.875996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.877427 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.877442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.878588 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.878594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.878746 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.878770 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 14:42:59.878878 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 14:42:59.879013 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":3,"thread_start_us":452,"threads_started":1}
I20260810 14:42:59.879984 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.873128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:57075 (local address 127.21.55.67:43893)
0810 14:42:59.873347 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:59.873353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:59.874736 (+ 1383us) server_negotiation.cc:407] Connection header received
0810 14:42:59.874972 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:59.874977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:59.875050 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:59.875152 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:42:59.875770 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.876488 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.878203 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.879039 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.879230 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.879306 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:59.879659 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 14:42:59.879805 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 14:42:59.880122 19716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.873271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:43893 (local address 127.21.55.65:57075)
0810 14:42:59.874616 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0810 14:42:59.874641 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:42:59.874808 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:42:59.875170 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:42:59.875174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:42:59.875198 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:42:59.875546 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.875579 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.876641 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.876647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.878025 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:42:59.878037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.879276 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.879283 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:42:59.879501 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.879530 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 14:42:59.879628 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 14:42:59.879759 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1179,"mutex_wait_us":63,"spinlock_wait_cycles":21888,"thread_start_us":920,"threads_started":1}
I20260810 14:42:59.881152 19380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd35f7d85ad4cb887e9efe75f4dbdba" candidate_uuid: "b258f2c1a4354048afbf073712ecd635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" is_pre_election: true
I20260810 14:42:59.881152 19590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd35f7d85ad4cb887e9efe75f4dbdba" candidate_uuid: "b258f2c1a4354048afbf073712ecd635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" is_pre_election: true
I20260810 14:42:59.881744 19590 raft_consensus.cc:2454] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b258f2c1a4354048afbf073712ecd635 in term 0.
I20260810 14:42:59.881744 19380 raft_consensus.cc:2454] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b258f2c1a4354048afbf073712ecd635 in term 0.
I20260810 14:42:59.882477 19101 leader_election.cc:302] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 92e1a7c0c6f14ea299c80d9167cccab3, b258f2c1a4354048afbf073712ecd635, e4f31a4b99b844ddba2c9cc108b2e441; no voters:
I20260810 14:42:59.882963 19714 raft_consensus.cc:2781] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:42:59.883128 19714 raft_consensus.cc:482] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:42:59.883256 19714 raft_consensus.cc:3037] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:59.885453 19714 raft_consensus.cc:504] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38429 } } peers { permanent_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43893 } } peers { permanent_uuid: "b258f2c1a4354048afbf073712ecd635" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33059 } }
I20260810 14:42:59.886092 19714 leader_election.cc:288] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [CANDIDATE]: Term 1 election: Requested vote from peers e4f31a4b99b844ddba2c9cc108b2e441 (127.21.55.66:38429), 92e1a7c0c6f14ea299c80d9167cccab3 (127.21.55.67:43893)
I20260810 14:42:59.886507 19380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd35f7d85ad4cb887e9efe75f4dbdba" candidate_uuid: "b258f2c1a4354048afbf073712ecd635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f31a4b99b844ddba2c9cc108b2e441"
I20260810 14:42:59.886623 19590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd35f7d85ad4cb887e9efe75f4dbdba" candidate_uuid: "b258f2c1a4354048afbf073712ecd635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92e1a7c0c6f14ea299c80d9167cccab3"
I20260810 14:42:59.886785 19380 raft_consensus.cc:3037] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:59.886821 19590 raft_consensus.cc:3037] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:42:59.888969 19590 raft_consensus.cc:2454] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b258f2c1a4354048afbf073712ecd635 in term 1.
I20260810 14:42:59.889196 19380 raft_consensus.cc:2454] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b258f2c1a4354048afbf073712ecd635 in term 1.
I20260810 14:42:59.889402 19103 leader_election.cc:302] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92e1a7c0c6f14ea299c80d9167cccab3, b258f2c1a4354048afbf073712ecd635; no voters:
I20260810 14:42:59.889701 19714 raft_consensus.cc:2781] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:42:59.890125 19714 raft_consensus.cc:686] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 1 LEADER]: Becoming Leader. State: Replica: b258f2c1a4354048afbf073712ecd635, State: Running, Role: LEADER
I20260810 14:42:59.890717 19714 consensus_queue.cc:237] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38429 } } peers { permanent_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43893 } } peers { permanent_uuid: "b258f2c1a4354048afbf073712ecd635" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33059 } }
I20260810 14:42:59.895231 18995 catalog_manager.cc:5495] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 reported cstate change: term changed from 0 to 1, leader changed from <none> to b258f2c1a4354048afbf073712ecd635 (127.21.55.65). New cstate: current_term: 1 leader_uuid: "b258f2c1a4354048afbf073712ecd635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b258f2c1a4354048afbf073712ecd635" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 33059 } health_report { overall_health: HEALTHY } } }
I20260810 14:42:59.916814 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.911182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40602 (local address 127.21.55.65:33059)
0810 14:42:59.911403 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:59.911408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:59.911427 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:42:59.911595 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:59.911599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:59.911692 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:59.911814 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:42:59.912215 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.912940 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.914414 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.914912 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.915016 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.915602 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:42:59.916162 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:42:59.916288 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:59.916455 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 14:42:59.916526 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 14:42:59.925832 19590 raft_consensus.cc:1261] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3 [term 1 FOLLOWER]: Refusing update from remote peer b258f2c1a4354048afbf073712ecd635: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:42:59.925832 19380 raft_consensus.cc:1261] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441 [term 1 FOLLOWER]: Refusing update from remote peer b258f2c1a4354048afbf073712ecd635: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:42:59.927356 19717 consensus_queue.cc:1035] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f31a4b99b844ddba2c9cc108b2e441" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 38429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:42:59.927862 19714 consensus_queue.cc:1035] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92e1a7c0c6f14ea299c80d9167cccab3" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 43893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:42:59.955550 18994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:42:59.955909 18994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:42:59.959815 18994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff228e9ccee843c3b285952cf92371df: Sending DeleteTablet for 3 replicas of tablet 2fd35f7d85ad4cb887e9efe75f4dbdba
I20260810 14:42:59.960986 19347 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fd35f7d85ad4cb887e9efe75f4dbdba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:59 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:57104
I20260810 14:42:59.961030 19561 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fd35f7d85ad4cb887e9efe75f4dbdba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:59 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:53454
I20260810 14:42:59.961535 19163 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fd35f7d85ad4cb887e9efe75f4dbdba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:42:59 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:40590
I20260810 14:42:59.965775 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 19081
I20260810 14:42:59.971647 19739 tablet_replica.cc:330] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635: stopping tablet replica
I20260810 14:42:59.971969 19737 tablet_replica.cc:330] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: stopping tablet replica
I20260810 14:42:59.972769 19737 raft_consensus.cc:2229] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:42:59.973747 19737 raft_consensus.cc:2258] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:42:59.974028 19739 raft_consensus.cc:2229] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:42:59.975847 19739 raft_consensus.cc:2258] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:42:59.977968 19737 ts_tablet_manager.cc:1905] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:59.979095 19739 ts_tablet_manager.cc:1905] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:59.985289 19738 tablet_replica.cc:330] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: stopping tablet replica
I20260810 14:42:59.985759 19738 raft_consensus.cc:2229] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:42:59.986301 19738 raft_consensus.cc:2258] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:42:59.987239 19737 ts_tablet_manager.cc:1918] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:42:59.987473 19737 log.cc:1196] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/2fd35f7d85ad4cb887e9efe75f4dbdba
I20260810 14:42:59.988154 19737 ts_tablet_manager.cc:1939] T 2fd35f7d85ad4cb887e9efe75f4dbdba P 92e1a7c0c6f14ea299c80d9167cccab3: Deleting consensus metadata
I20260810 14:42:59.990181 18981 catalog_manager.cc:4841] TS 92e1a7c0c6f14ea299c80d9167cccab3 (127.21.55.67:43893): tablet 2fd35f7d85ad4cb887e9efe75f4dbdba (table test-table [id=cd516f2397c5409695861107d35f1019]) successfully deleted
I20260810 14:42:59.991787 19739 ts_tablet_manager.cc:1918] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:42:59.991999 19739 log.cc:1196] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/2fd35f7d85ad4cb887e9efe75f4dbdba
I20260810 14:42:59.992581 19739 ts_tablet_manager.cc:1939] T 2fd35f7d85ad4cb887e9efe75f4dbdba P b258f2c1a4354048afbf073712ecd635: Deleting consensus metadata
Aug 10 14:42:59 dist-test-slave-h6n0 krb5kdc[18937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:42:59.993314 19738 ts_tablet_manager.cc:1905] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:42:59.994423 18976 catalog_manager.cc:4841] TS b258f2c1a4354048afbf073712ecd635 (127.21.55.65:33059): tablet 2fd35f7d85ad4cb887e9efe75f4dbdba (table test-table [id=cd516f2397c5409695861107d35f1019]) successfully deleted
I20260810 14:43:00.001066 19738 ts_tablet_manager.cc:1918] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:43:00.001271 19738 log.cc:1196] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/2fd35f7d85ad4cb887e9efe75f4dbdba
I20260810 14:43:00.001349 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:42:59.966779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40616 (local address 127.21.55.65:33059)
0810 14:42:59.967018 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 14:42:59.967028 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:42:59.981008 (+ 13980us) server_negotiation.cc:407] Connection header received
0810 14:42:59.981473 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:42:59.981479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:42:59.981596 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:42:59.981695 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:42:59.983629 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.984875 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.986874 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:42:59.987618 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:42:59.987718 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:42:59.994091 (+ 6373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:42:59.994106 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:42:59.994127 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:42:59.994176 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:42:59.997555 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:59.998292 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:59.998299 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:59.998301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:59.998459 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:42:59.998962 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:42:59.998967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:42:59.998968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:42:59.999306 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:42:59.999647 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 14:42:59.999924 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 14:43:00.000163 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 14:43:00.001864 19738 ts_tablet_manager.cc:1939] T 2fd35f7d85ad4cb887e9efe75f4dbdba P e4f31a4b99b844ddba2c9cc108b2e441: Deleting consensus metadata
I20260810 14:43:00.002710 19252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:00.003857 18976 catalog_manager.cc:4841] TS e4f31a4b99b844ddba2c9cc108b2e441 (127.21.55.66:38429): tablet 2fd35f7d85ad4cb887e9efe75f4dbdba (table test-table [id=cd516f2397c5409695861107d35f1019]) successfully deleted
I20260810 14:43:00.109433 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 19081
I20260810 14:43:00.139143 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 19269
Aug 10 14:43:00 dist-test-slave-h6n0 krb5kdc[18937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:43:00.157405 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:00.140113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57118 (local address 127.21.55.66:38429)
0810 14:43:00.140329 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:00.140334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:00.140354 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:43:00.140613 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:00.140620 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:00.140743 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:00.140852 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:00.142354 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:00.143252 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:00.144666 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:00.145173 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:00.145283 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:00.150090 (+ 4807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:00.150104 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:00.150134 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:00.150169 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:00.153923 (+ 3754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:00.154905 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:00.154916 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:00.154919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:00.155214 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:00.155735 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:00.155739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:00.155742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:00.156186 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:00.156424 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:00.156761 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 14:43:00.156929 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 14:43:00.158159 19448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:00.251729 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 19269
I20260810 14:43:00.284878 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 19467
Aug 10 14:43:00 dist-test-slave-h6n0 krb5kdc[18937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:43:00.301265 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:00.285700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.21.55.67:43893)
0810 14:43:00.285914 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:00.285921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:00.285951 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:43:00.286084 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:00.286088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:00.286156 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:00.286287 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:00.287668 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:00.288405 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:00.289667 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:00.290067 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:00.290207 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:00.294990 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:00.295004 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:00.295020 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:00.295051 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:00.298027 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:00.298930 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:00.298938 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:00.298941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:00.299096 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:00.299557 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:00.299561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:00.299563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:00.299949 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:00.300285 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:00.300644 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 14:43:00.300892 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 14:43:00.302105 19654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:00.406960 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 19467
I20260810 14:43:00.437791 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 18955
I20260810 14:43:00.439595 19040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:00.559917 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 18955
2026-08-10T14:43:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4976 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T14:43:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:00Z Disabled control of system clock
I20260810 14:43:00.664672 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:40393
--webserver_interface=127.21.55.126
--webserver_port=0
--builtin_ntp_servers=127.21.55.84:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:40393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:43:01.008872 19753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:01.009181 19753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:01.009439 19753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:01.016614 19753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:01.016711 19753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:01.016804 19753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:01.016916 19753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:01.017002 19753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:01.025995 19753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:40393
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:40393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:01.027354 19753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:01.029541 19753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:01.042274 19762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:01.042192 19765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:01.042300 19763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:01.043570 19753 server_base.cc:1034] running on GCE node
I20260810 14:43:01.044919 19753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:01.046736 19753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:01.048094 19753 hybrid_clock.cc:648] HybridClock initialized: now 1786372981047958 us; error 58 us; skew 500 ppm
I20260810 14:43:01.048516 19753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:43:01.051275 19753 webserver.cc:460] Webserver started at http://127.21.55.126:40641/ using document root <none> and password file <none>
I20260810 14:43:01.051964 19753 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:01.052101 19753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:01.052433 19753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:01.055368 19753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "45d114332fff407fafe3dec8b2d30ea1"
format_stamp: "Formatted at 2026-08-10 14:43:01 on dist-test-slave-h6n0"
server_key: "9e98e4c7ef161a44c9d977808ae91022"
server_key_iv: "6bfc1ce02901b70ece3b3c0f7b6a80ca"
server_key_version: "encryptionkey@0"
I20260810 14:43:01.056339 19753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "45d114332fff407fafe3dec8b2d30ea1"
format_stamp: "Formatted at 2026-08-10 14:43:01 on dist-test-slave-h6n0"
server_key: "9e98e4c7ef161a44c9d977808ae91022"
server_key_iv: "6bfc1ce02901b70ece3b3c0f7b6a80ca"
server_key_version: "encryptionkey@0"
I20260810 14:43:01.061590 19753 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 14:43:01.065232 19773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:01.066612 19753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:43:01.066797 19753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "45d114332fff407fafe3dec8b2d30ea1"
format_stamp: "Formatted at 2026-08-10 14:43:01 on dist-test-slave-h6n0"
server_key: "9e98e4c7ef161a44c9d977808ae91022"
server_key_iv: "6bfc1ce02901b70ece3b3c0f7b6a80ca"
server_key_version: "encryptionkey@0"
I20260810 14:43:01.067022 19753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:01.087217 19753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:01.088379 19753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:01.088737 19753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:01.111845 19753 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:40393
I20260810 14:43:01.111867 19850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:40393 every 8 connection(s)
I20260810 14:43:01.113232 19753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:01.117166 19854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:01.121093 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 19753
I20260810 14:43:01.121397 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:01.122161 21725 external_mini_cluster.cc:1428] Setting key b4b2ceedc53c306ee3f35daaa0c33a08
I20260810 14:43:01.134063 19854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1: Bootstrap starting.
I20260810 14:43:01.138229 19854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:01.139597 19854 log.cc:824] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:01.139833 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:01.124965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.21.55.126:40393)
0810 14:43:01.126241 (+ 1276us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:01.126286 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:01.126353 (+ 67us) server_negotiation.cc:407] Connection header received
0810 14:43:01.128580 (+ 2227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:01.128613 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:01.129542 (+ 929us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:01.130037 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:01.130691 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:01.131975 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:01.133684 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:01.135063 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:01.135245 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:01.136975 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:01.136987 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:01.137008 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:01.137830 (+ 822us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:01.137941 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:01.137966 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:01.138053 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:01.138429 (+ 376us) server_negotiation.cc:299] Negotiation successful
0810 14:43:01.138820 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":366,"threads_started":1}
I20260810 14:43:01.142525 19854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1: No bootstrap required, opened a new log
I20260810 14:43:01.150190 19854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d114332fff407fafe3dec8b2d30ea1" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40393 } }
I20260810 14:43:01.150593 19854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:01.150694 19854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d114332fff407fafe3dec8b2d30ea1, State: Initialized, Role: FOLLOWER
I20260810 14:43:01.151185 19854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d114332fff407fafe3dec8b2d30ea1" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40393 } }
I20260810 14:43:01.151393 19854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:01.151484 19854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:01.151605 19854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:01.153672 19854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d114332fff407fafe3dec8b2d30ea1" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40393 } }
I20260810 14:43:01.154107 19854 leader_election.cc:302] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45d114332fff407fafe3dec8b2d30ea1; no voters:
I20260810 14:43:01.154881 19854 leader_election.cc:288] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:01.155228 19865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:01.156579 19865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [term 1 LEADER]: Becoming Leader. State: Replica: 45d114332fff407fafe3dec8b2d30ea1, State: Running, Role: LEADER
I20260810 14:43:01.157155 19854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:01.157083 19865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d114332fff407fafe3dec8b2d30ea1" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40393 } }
I20260810 14:43:01.160055 19867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45d114332fff407fafe3dec8b2d30ea1. Latest consensus state: current_term: 1 leader_uuid: "45d114332fff407fafe3dec8b2d30ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d114332fff407fafe3dec8b2d30ea1" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40393 } } }
I20260810 14:43:01.161065 19867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:01.160612 19866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45d114332fff407fafe3dec8b2d30ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d114332fff407fafe3dec8b2d30ea1" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40393 } } }
I20260810 14:43:01.161731 19866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:01.162287 19879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:01.171868 19879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:01.180639 19879 catalog_manager.cc:1347] Generated new cluster ID: c4f549f89df3492baab690a701a8b9d7
I20260810 14:43:01.180883 19879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:01.191681 19879 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:01.193473 19879 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:01.210712 19879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1: Generated new TSK 0
I20260810 14:43:01.211817 19879 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:01.227991 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40393
--builtin_ntp_servers=127.21.55.84:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:43:01.581427 19896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:01.581750 19896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:01.581925 19896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:01.589377 19896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:01.589535 19896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:01.589725 19896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:43:01.599066 19896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:01.600364 19896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:01.602160 19896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:01.615708 19905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:01.615906 19907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:01.617265 19896 server_base.cc:1034] running on GCE node
W20260810 14:43:01.618343 19904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:01.619151 19896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:01.620486 19896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:01.621843 19896 hybrid_clock.cc:648] HybridClock initialized: now 1786372981621728 us; error 50 us; skew 500 ppm
I20260810 14:43:01.622460 19896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:43:01.625798 19896 webserver.cc:460] Webserver started at http://127.21.55.65:36107/ using document root <none> and password file <none>
I20260810 14:43:01.626847 19896 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:01.626966 19896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:01.627374 19896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:01.631836 19896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "37f0413553c447628ce8d71f826a4f15"
format_stamp: "Formatted at 2026-08-10 14:43:01 on dist-test-slave-h6n0"
server_key: "ed1a8da7a829cc2708cbbbd9072bafd7"
server_key_iv: "207aa517099406af599522da496b7eca"
server_key_version: "encryptionkey@0"
I20260810 14:43:01.632918 19896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "37f0413553c447628ce8d71f826a4f15"
format_stamp: "Formatted at 2026-08-10 14:43:01 on dist-test-slave-h6n0"
server_key: "ed1a8da7a829cc2708cbbbd9072bafd7"
server_key_iv: "207aa517099406af599522da496b7eca"
server_key_version: "encryptionkey@0"
I20260810 14:43:01.639106 19896 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 14:43:01.642899 19917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:01.644315 19896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:43:01.644462 19896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "37f0413553c447628ce8d71f826a4f15"
format_stamp: "Formatted at 2026-08-10 14:43:01 on dist-test-slave-h6n0"
server_key: "ed1a8da7a829cc2708cbbbd9072bafd7"
server_key_iv: "207aa517099406af599522da496b7eca"
server_key_version: "encryptionkey@0"
I20260810 14:43:01.644729 19896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:01.681794 19896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:01.682760 19896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:01.683111 19896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:01.684320 19896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:43:01.686010 19896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:43:01.686120 19896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:01.686240 19896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:43:01.686347 19896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:01.726398 19896 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:35387
I20260810 14:43:01.726598 20087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:35387 every 8 connection(s)
I20260810 14:43:01.727790 19896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:43:01.735070 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 19896
I20260810 14:43:01.735278 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:43:01.735843 21725 external_mini_cluster.cc:1428] Setting key c730a78d8203e60d22e191f32d0185fd
I20260810 14:43:01.738914 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40393
--builtin_ntp_servers=127.21.55.84:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:43:01.744467 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:01.731450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:39395 (local address 127.21.55.126:40393)
0810 14:43:01.732053 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:01.732059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:01.733444 (+ 1385us) server_negotiation.cc:407] Connection header received
0810 14:43:01.736712 (+ 3268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:01.736719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:01.736827 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:01.736964 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:01.738133 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:01.738950 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:01.741544 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:01.741957 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:01.742052 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:01.743711 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:01.743715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:01.743719 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:01.743736 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:01.743774 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:01.743786 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:01.743869 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:01.744119 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 14:43:01.744205 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":308,"threads_started":1}
I20260810 14:43:01.745929 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:01.732001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:40393 (local address 127.21.55.65:39395)
0810 14:43:01.733220 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:01.733272 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:01.736317 (+ 3045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:01.737202 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:01.737210 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:01.737306 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:01.737934 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:01.737961 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:01.739159 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:01.739168 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:01.741374 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:01.741392 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:01.742188 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:01.742198 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:01.742547 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:01.743333 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:43:01.743339 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:01.743394 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:43:01.743517 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:01.743899 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:01.743912 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:01.743922 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:43:01.744104 (+ 182us) client_negotiation.cc:240] Negotiation successful
0810 14:43:01.744538 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":356,"threads_started":1}
I20260810 14:43:01.747973 20088 heartbeater.cc:347] Connected to a master server at 127.21.55.126:40393
I20260810 14:43:01.748340 20088 heartbeater.cc:464] Registering TS with master...
I20260810 14:43:01.749234 20088 heartbeater.cc:511] Master 127.21.55.126:40393 requested a full tablet report, sending...
I20260810 14:43:01.751518 19794 ts_manager.cc:194] Registered new tserver with Master: 37f0413553c447628ce8d71f826a4f15 (127.21.55.65:35387)
I20260810 14:43:01.753418 19794 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.65:39395
W20260810 14:43:02.095585 20093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:02.095861 20093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:02.096004 20093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:02.103014 20093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:02.103099 20093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:02.103308 20093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:43:02.111936 20093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:02.113168 20093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:02.114858 20093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:02.127676 20102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:02.127714 20100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:02.129573 20098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:02.129772 20093 server_base.cc:1034] running on GCE node
I20260810 14:43:02.130467 20093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:02.131472 20093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:02.132839 20093 hybrid_clock.cc:648] HybridClock initialized: now 1786372982132740 us; error 47 us; skew 500 ppm
I20260810 14:43:02.133456 20093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:43:02.136112 20093 webserver.cc:460] Webserver started at http://127.21.55.66:42419/ using document root <none> and password file <none>
I20260810 14:43:02.136776 20093 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:02.136873 20093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:02.137123 20093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:02.140004 20093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "232de782c610445c819164b562a9eacc"
format_stamp: "Formatted at 2026-08-10 14:43:02 on dist-test-slave-h6n0"
server_key: "d3f4369fcd7d4a0799f80dbd400891af"
server_key_iv: "b6f2c4181db293b459e3e6cdb98f08d9"
server_key_version: "encryptionkey@0"
I20260810 14:43:02.140746 20093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "232de782c610445c819164b562a9eacc"
format_stamp: "Formatted at 2026-08-10 14:43:02 on dist-test-slave-h6n0"
server_key: "d3f4369fcd7d4a0799f80dbd400891af"
server_key_iv: "b6f2c4181db293b459e3e6cdb98f08d9"
server_key_version: "encryptionkey@0"
I20260810 14:43:02.146448 20093 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:43:02.151789 20111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:02.153704 20093 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260810 14:43:02.153946 20093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "232de782c610445c819164b562a9eacc"
format_stamp: "Formatted at 2026-08-10 14:43:02 on dist-test-slave-h6n0"
server_key: "d3f4369fcd7d4a0799f80dbd400891af"
server_key_iv: "b6f2c4181db293b459e3e6cdb98f08d9"
server_key_version: "encryptionkey@0"
I20260810 14:43:02.154281 20093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:02.176522 20093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:02.177433 20093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:02.177699 20093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:02.178776 20093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:43:02.180361 20093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:43:02.180461 20093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:02.180537 20093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:43:02.180600 20093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:02.218997 20093 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:39657
I20260810 14:43:02.219122 20289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:39657 every 8 connection(s)
I20260810 14:43:02.220310 20093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:43:02.224639 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 20093
I20260810 14:43:02.224831 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:43:02.225289 21725 external_mini_cluster.cc:1428] Setting key f9de1cb5e757602db3d227976a22bb85
I20260810 14:43:02.228683 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40393
--builtin_ntp_servers=127.21.55.84:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:43:02.233711 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.222996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:50805 (local address 127.21.55.126:40393)
0810 14:43:02.223186 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:02.223191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:02.224422 (+ 1231us) server_negotiation.cc:407] Connection header received
0810 14:43:02.226820 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:02.226827 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:02.226938 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:02.227127 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:02.228317 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.229170 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.230925 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.231325 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.231468 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.232871 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:02.232876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:02.232880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:02.232899 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:02.232930 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:02.232942 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:02.233051 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:02.233288 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 14:43:02.233386 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:43:02.234777 20297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.223376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:40393 (local address 127.21.55.66:50805)
0810 14:43:02.224250 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:02.224292 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:02.226274 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:02.227427 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:02.227440 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:02.227555 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:02.228133 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.228150 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.229344 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.229348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.230770 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.230783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.231567 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.231575 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.231800 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.232541 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:43:02.232544 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:02.232582 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:43:02.232663 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:02.233051 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:02.233065 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:02.233075 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:43:02.233266 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 14:43:02.233658 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":278,"threads_started":1}
I20260810 14:43:02.236584 20292 heartbeater.cc:347] Connected to a master server at 127.21.55.126:40393
I20260810 14:43:02.236924 20292 heartbeater.cc:464] Registering TS with master...
I20260810 14:43:02.237671 20292 heartbeater.cc:511] Master 127.21.55.126:40393 requested a full tablet report, sending...
I20260810 14:43:02.239236 19794 ts_manager.cc:194] Registered new tserver with Master: 232de782c610445c819164b562a9eacc (127.21.55.66:39657)
I20260810 14:43:02.240427 19794 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.66:50805
W20260810 14:43:02.580049 20298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:02.580379 20298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:02.580560 20298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:02.587721 20298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:02.587858 20298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:02.588048 20298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:43:02.597445 20298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:02.598912 20298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:02.600983 20298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:02.614490 20309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:02.614871 20308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:02.615278 20312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:02.616214 20298 server_base.cc:1034] running on GCE node
I20260810 14:43:02.616935 20298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:02.617866 20298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:02.619086 20298 hybrid_clock.cc:648] HybridClock initialized: now 1786372982619048 us; error 38 us; skew 500 ppm
I20260810 14:43:02.619366 20298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:43:02.621584 20298 webserver.cc:460] Webserver started at http://127.21.55.67:37697/ using document root <none> and password file <none>
I20260810 14:43:02.622227 20298 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:02.622316 20298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:02.622712 20298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:02.625432 20298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc"
format_stamp: "Formatted at 2026-08-10 14:43:02 on dist-test-slave-h6n0"
server_key: "62520215eaab9c1d02e75b74af4cf923"
server_key_iv: "b6481029077647286037082f31a54cf7"
server_key_version: "encryptionkey@0"
I20260810 14:43:02.626159 20298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc"
format_stamp: "Formatted at 2026-08-10 14:43:02 on dist-test-slave-h6n0"
server_key: "62520215eaab9c1d02e75b74af4cf923"
server_key_iv: "b6481029077647286037082f31a54cf7"
server_key_version: "encryptionkey@0"
I20260810 14:43:02.631374 20298 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:43:02.634778 20319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:02.636230 20298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:43:02.636366 20298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc"
format_stamp: "Formatted at 2026-08-10 14:43:02 on dist-test-slave-h6n0"
server_key: "62520215eaab9c1d02e75b74af4cf923"
server_key_iv: "b6481029077647286037082f31a54cf7"
server_key_version: "encryptionkey@0"
I20260810 14:43:02.636600 20298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:02.687304 20298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:02.688212 20298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:02.688544 20298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:02.689590 20298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:43:02.691154 20298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:43:02.691247 20298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:02.691407 20298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:43:02.691505 20298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:02.728876 20298 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:36979
I20260810 14:43:02.728936 20485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:36979 every 8 connection(s)
I20260810 14:43:02.730399 20298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:43:02.734938 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 20298
I20260810 14:43:02.735181 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:43:02.735810 21725 external_mini_cluster.cc:1428] Setting key 4878283fc081b63728cd715e8566d309
I20260810 14:43:02.744201 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.733185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:44467 (local address 127.21.55.126:40393)
0810 14:43:02.733414 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:02.733422 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:02.734890 (+ 1468us) server_negotiation.cc:407] Connection header received
0810 14:43:02.737328 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:02.737334 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:02.737427 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:02.737546 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:02.738967 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.739745 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.741375 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.741781 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.741931 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.743373 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:02.743378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:02.743382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:02.743399 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:02.743430 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:02.743442 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:02.743550 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:02.743783 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 14:43:02.743877 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:43:02.745116 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.733648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:40393 (local address 127.21.55.67:44467)
0810 14:43:02.734689 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:02.734742 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:02.737011 (+ 2269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:02.737808 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:02.737828 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:02.737945 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:02.738782 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.738803 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.739912 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.739918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.741205 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.741218 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.742034 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.742042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.742287 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.743042 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:43:02.743044 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:02.743083 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:43:02.743160 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:02.743552 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:02.743566 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:02.743575 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 14:43:02.743753 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 14:43:02.744136 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":284,"threads_started":1}
I20260810 14:43:02.746519 20486 heartbeater.cc:347] Connected to a master server at 127.21.55.126:40393
I20260810 14:43:02.746847 20486 heartbeater.cc:464] Registering TS with master...
I20260810 14:43:02.747575 20486 heartbeater.cc:511] Master 127.21.55.126:40393 requested a full tablet report, sending...
I20260810 14:43:02.748936 19794 ts_manager.cc:194] Registered new tserver with Master: 4e16dce3be2f4f598ee87e8da5c2d0cc (127.21.55.67:36979)
I20260810 14:43:02.749864 19794 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.67:44467
I20260810 14:43:02.752480 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:43:02.756304 20088 heartbeater.cc:503] Master 127.21.55.126:40393 was elected leader, sending a full tablet report...
I20260810 14:43:02.763190 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.756734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45998 (local address 127.21.55.126:40393)
0810 14:43:02.756927 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:02.756936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:02.757472 (+ 536us) server_negotiation.cc:407] Connection header received
0810 14:43:02.757642 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:02.757645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:02.757743 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:02.757836 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:02.758424 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.759168 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.760644 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.761081 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.761183 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.762374 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:02.762378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:02.762382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:02.762399 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:02.762441 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:02.762452 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:02.762656 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:02.762814 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 14:43:02.762881 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:43:02.767234 19794 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:43:02.773937 19794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:43:02.803995 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.787133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.21.55.65:35387)
0810 14:43:02.791895 (+ 4762us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:02.791909 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:02.793159 (+ 1250us) server_negotiation.cc:407] Connection header received
0810 14:43:02.793281 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:02.793292 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:02.793556 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:02.793757 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:02.794861 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.796936 (+ 2075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.800948 (+ 4012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.801863 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.802027 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.802334 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:02.803470 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0810 14:43:02.803649 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4570,"thread_start_us":4394,"threads_started":1}
I20260810 14:43:02.804484 20505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.786482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:35387 (local address 127.0.0.1:54658)
0810 14:43:02.792815 (+ 6333us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:02.792840 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:02.793008 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:02.794058 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:02.794066 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:02.794101 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:02.794639 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.794655 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.797165 (+ 2510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.797174 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.800753 (+ 3579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.800766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.802114 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.802121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.802803 (+ 682us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.802872 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 14:43:02.803007 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 14:43:02.803199 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6139,"mutex_wait_us":312,"thread_start_us":947,"threads_started":1}
I20260810 14:43:02.804852 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.786267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:36979 (local address 127.0.0.1:38274)
0810 14:43:02.787196 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:02.787245 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:02.787403 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:02.793541 (+ 6138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:02.793553 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:02.793593 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:02.794078 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.794092 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.797292 (+ 3200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.797304 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.802186 (+ 4882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.802201 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.803918 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.803925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.804113 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.804152 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 14:43:02.804243 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 14:43:02.804376 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":25,"thread_start_us":13227,"threads_started":1}
I20260810 14:43:02.804819 20501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.786560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38274 (local address 127.21.55.67:36979)
0810 14:43:02.792414 (+ 5854us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:02.792422 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:02.792460 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:43:02.792567 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:02.792577 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:02.792863 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:02.793136 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:02.794343 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.796853 (+ 2510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.802417 (+ 5564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.803690 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.803845 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.804120 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:02.804294 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 14:43:02.804450 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5689,"thread_start_us":5517,"threads_started":1}
I20260810 14:43:02.808568 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.791854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:39657 (local address 127.0.0.1:39404)
0810 14:43:02.799713 (+ 7859us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:02.799728 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:02.799832 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:02.800342 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:02.800348 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:02.800368 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:02.800690 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.800698 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.802470 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.802475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.805198 (+ 2723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.805211 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.806969 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.806975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.807145 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.807182 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 14:43:02.807274 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 14:43:02.807389 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7765,"thread_start_us":728,"threads_started":1}
I20260810 14:43:02.809043 20506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.792392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.21.55.66:39657)
0810 14:43:02.798024 (+ 5632us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:02.798033 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:02.799836 (+ 1803us) server_negotiation.cc:407] Connection header received
0810 14:43:02.799936 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:02.799942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:02.800146 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:02.800342 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:02.800866 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.802344 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.805364 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.806759 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.806902 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.807142 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:02.808497 (+ 1355us) server_negotiation.cc:299] Negotiation successful
0810 14:43:02.808689 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5457,"thread_start_us":5611,"threads_started":1}
I20260810 14:43:02.808771 19987 tablet_service.cc:1467] Processing CreateTablet for tablet be2e654e699e4574844e187a173da8c3 (DEFAULT_TABLE table=test-table [id=3270c30d6c8641078ccb3a4bd141bc35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:43:02.809072 20396 tablet_service.cc:1467] Processing CreateTablet for tablet be2e654e699e4574844e187a173da8c3 (DEFAULT_TABLE table=test-table [id=3270c30d6c8641078ccb3a4bd141bc35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:43:02.810294 19987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2e654e699e4574844e187a173da8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:02.810312 20396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2e654e699e4574844e187a173da8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:02.811705 20185 tablet_service.cc:1467] Processing CreateTablet for tablet be2e654e699e4574844e187a173da8c3 (DEFAULT_TABLE table=test-table [id=3270c30d6c8641078ccb3a4bd141bc35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:43:02.812804 20185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2e654e699e4574844e187a173da8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:02.823988 20507 tablet_bootstrap.cc:492] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: Bootstrap starting.
I20260810 14:43:02.824924 20510 tablet_bootstrap.cc:492] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: Bootstrap starting.
I20260810 14:43:02.825711 20508 tablet_bootstrap.cc:492] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: Bootstrap starting.
I20260810 14:43:02.829253 20507 tablet_bootstrap.cc:654] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:02.829365 20510 tablet_bootstrap.cc:654] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:02.829520 20508 tablet_bootstrap.cc:654] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:02.830714 20510 log.cc:824] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:02.830682 20507 log.cc:824] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:02.830731 20508 log.cc:824] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:02.833896 20508 tablet_bootstrap.cc:492] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: No bootstrap required, opened a new log
I20260810 14:43:02.834193 20508 ts_tablet_manager.cc:1397] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 14:43:02.834995 20510 tablet_bootstrap.cc:492] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: No bootstrap required, opened a new log
I20260810 14:43:02.835510 20510 ts_tablet_manager.cc:1397] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260810 14:43:02.836364 20507 tablet_bootstrap.cc:492] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: No bootstrap required, opened a new log
I20260810 14:43:02.836730 20507 ts_tablet_manager.cc:1397] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260810 14:43:02.842711 20508 raft_consensus.cc:348] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36979 } } peers { permanent_uuid: "37f0413553c447628ce8d71f826a4f15" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35387 } } peers { permanent_uuid: "232de782c610445c819164b562a9eacc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39657 } }
I20260810 14:43:02.843152 20508 raft_consensus.cc:374] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:02.843269 20508 raft_consensus.cc:729] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e16dce3be2f4f598ee87e8da5c2d0cc, State: Initialized, Role: FOLLOWER
I20260810 14:43:02.843945 20508 consensus_queue.cc:260] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36979 } } peers { permanent_uuid: "37f0413553c447628ce8d71f826a4f15" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35387 } } peers { permanent_uuid: "232de782c610445c819164b562a9eacc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39657 } }
I20260810 14:43:02.845312 20486 heartbeater.cc:503] Master 127.21.55.126:40393 was elected leader, sending a full tablet report...
I20260810 14:43:02.845774 20508 ts_tablet_manager.cc:1428] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: Time spent starting tablet: real 0.011s user 0.005s sys 0.007s
I20260810 14:43:02.849031 20510 raft_consensus.cc:348] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36979 } } peers { permanent_uuid: "37f0413553c447628ce8d71f826a4f15" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35387 } } peers { permanent_uuid: "232de782c610445c819164b562a9eacc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39657 } }
I20260810 14:43:02.849617 20510 raft_consensus.cc:374] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:02.849776 20510 raft_consensus.cc:729] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37f0413553c447628ce8d71f826a4f15, State: Initialized, Role: FOLLOWER
I20260810 14:43:02.849766 20507 raft_consensus.cc:348] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36979 } } peers { permanent_uuid: "37f0413553c447628ce8d71f826a4f15" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35387 } } peers { permanent_uuid: "232de782c610445c819164b562a9eacc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39657 } }
I20260810 14:43:02.850361 20507 raft_consensus.cc:374] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:02.850487 20507 raft_consensus.cc:729] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 232de782c610445c819164b562a9eacc, State: Initialized, Role: FOLLOWER
I20260810 14:43:02.850757 20510 consensus_queue.cc:260] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36979 } } peers { permanent_uuid: "37f0413553c447628ce8d71f826a4f15" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35387 } } peers { permanent_uuid: "232de782c610445c819164b562a9eacc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39657 } }
I20260810 14:43:02.851275 20507 consensus_queue.cc:260] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36979 } } peers { permanent_uuid: "37f0413553c447628ce8d71f826a4f15" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35387 } } peers { permanent_uuid: "232de782c610445c819164b562a9eacc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39657 } }
I20260810 14:43:02.853210 20510 ts_tablet_manager.cc:1428] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: Time spent starting tablet: real 0.018s user 0.009s sys 0.007s
I20260810 14:43:02.854437 20292 heartbeater.cc:503] Master 127.21.55.126:40393 was elected leader, sending a full tablet report...
I20260810 14:43:02.855214 20507 ts_tablet_manager.cc:1428] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: Time spent starting tablet: real 0.018s user 0.015s sys 0.003s
W20260810 14:43:02.972749 20296 tablet.cc:2369] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:43:02.975749 20516 raft_consensus.cc:482] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:43:02.976046 20516 raft_consensus.cc:504] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36979 } } peers { permanent_uuid: "37f0413553c447628ce8d71f826a4f15" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35387 } } peers { permanent_uuid: "232de782c610445c819164b562a9eacc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39657 } }
I20260810 14:43:02.977852 20516 leader_election.cc:288] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37f0413553c447628ce8d71f826a4f15 (127.21.55.65:35387), 232de782c610445c819164b562a9eacc (127.21.55.66:39657)
W20260810 14:43:02.980600 20089 tablet.cc:2369] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:43:02.982262 20487 tablet.cc:2369] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:43:02.985598 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.978317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:34265 (local address 127.21.55.65:35387)
0810 14:43:02.978573 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:02.978581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:02.979327 (+ 746us) server_negotiation.cc:407] Connection header received
0810 14:43:02.979518 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:02.979525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:02.979598 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:02.979701 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:02.980233 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.981354 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.983695 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.984626 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.984781 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.984847 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:02.985232 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 14:43:02.985352 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 14:43:02.985710 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.978398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:35387 (local address 127.21.55.67:34265)
0810 14:43:02.979225 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:02.979238 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:02.979373 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:02.979726 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:02.979730 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:02.979753 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:02.980083 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.980091 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.981511 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.981517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.983514 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.983526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.984780 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.984786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.985042 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.985088 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 14:43:02.985193 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 14:43:02.985325 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":39,"thread_start_us":504,"threads_started":1}
I20260810 14:43:02.985973 20506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.978465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:48841 (local address 127.21.55.66:39657)
0810 14:43:02.978675 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:02.978681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:02.978919 (+ 238us) server_negotiation.cc:407] Connection header received
0810 14:43:02.979169 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:02.979173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:02.979253 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:02.979378 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:02.979822 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.980539 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.983103 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.984515 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.984701 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.984781 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:02.985464 (+ 683us) server_negotiation.cc:299] Negotiation successful
0810 14:43:02.985635 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:43:02.986019 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:02.978332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:39657 (local address 127.21.55.67:48841)
0810 14:43:02.978774 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:02.978797 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:02.978962 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:02.979397 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:02.979402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:02.979432 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:02.979647 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.979657 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.980684 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.980693 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.982618 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:02.982659 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:02.984701 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:02.984709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:02.985177 (+ 468us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:02.985223 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 14:43:02.985366 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 14:43:02.985552 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":61}
I20260810 14:43:02.987033 20015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2e654e699e4574844e187a173da8c3" candidate_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37f0413553c447628ce8d71f826a4f15" is_pre_election: true
I20260810 14:43:02.987536 20015 raft_consensus.cc:2454] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e16dce3be2f4f598ee87e8da5c2d0cc in term 0.
I20260810 14:43:02.987413 20222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2e654e699e4574844e187a173da8c3" candidate_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "232de782c610445c819164b562a9eacc" is_pre_election: true
I20260810 14:43:02.987962 20222 raft_consensus.cc:2454] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e16dce3be2f4f598ee87e8da5c2d0cc in term 0.
I20260810 14:43:02.988235 20325 leader_election.cc:302] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37f0413553c447628ce8d71f826a4f15, 4e16dce3be2f4f598ee87e8da5c2d0cc; no voters:
I20260810 14:43:02.988641 20516 raft_consensus.cc:2781] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:43:02.988798 20516 raft_consensus.cc:482] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:43:02.988930 20516 raft_consensus.cc:3037] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:02.991219 20516 raft_consensus.cc:504] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36979 } } peers { permanent_uuid: "37f0413553c447628ce8d71f826a4f15" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35387 } } peers { permanent_uuid: "232de782c610445c819164b562a9eacc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39657 } }
I20260810 14:43:02.991886 20516 leader_election.cc:288] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [CANDIDATE]: Term 1 election: Requested vote from peers 37f0413553c447628ce8d71f826a4f15 (127.21.55.65:35387), 232de782c610445c819164b562a9eacc (127.21.55.66:39657)
I20260810 14:43:02.992300 20015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2e654e699e4574844e187a173da8c3" candidate_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37f0413553c447628ce8d71f826a4f15"
I20260810 14:43:02.992405 20222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2e654e699e4574844e187a173da8c3" candidate_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "232de782c610445c819164b562a9eacc"
I20260810 14:43:02.992514 20015 raft_consensus.cc:3037] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:02.992651 20222 raft_consensus.cc:3037] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:02.994808 20015 raft_consensus.cc:2454] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e16dce3be2f4f598ee87e8da5c2d0cc in term 1.
I20260810 14:43:02.995023 20222 raft_consensus.cc:2454] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e16dce3be2f4f598ee87e8da5c2d0cc in term 1.
I20260810 14:43:02.995210 20325 leader_election.cc:302] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37f0413553c447628ce8d71f826a4f15, 4e16dce3be2f4f598ee87e8da5c2d0cc; no voters:
I20260810 14:43:02.995514 20516 raft_consensus.cc:2781] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:02.995920 20516 raft_consensus.cc:686] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 1 LEADER]: Becoming Leader. State: Replica: 4e16dce3be2f4f598ee87e8da5c2d0cc, State: Running, Role: LEADER
I20260810 14:43:02.996371 20516 consensus_queue.cc:237] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36979 } } peers { permanent_uuid: "37f0413553c447628ce8d71f826a4f15" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35387 } } peers { permanent_uuid: "232de782c610445c819164b562a9eacc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39657 } }
I20260810 14:43:03.001157 19794 catalog_manager.cc:5495] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e16dce3be2f4f598ee87e8da5c2d0cc (127.21.55.67). New cstate: current_term: 1 leader_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e16dce3be2f4f598ee87e8da5c2d0cc" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 36979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37f0413553c447628ce8d71f826a4f15" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "232de782c610445c819164b562a9eacc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39657 } health_report { overall_health: UNKNOWN } } }
I20260810 14:43:03.046113 20501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:03.039365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38286 (local address 127.21.55.67:36979)
0810 14:43:03.039686 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:03.039693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:03.039714 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:43:03.039854 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:03.039858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:03.039948 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:03.040120 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:43:03.040522 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:03.041328 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:03.043026 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:03.043599 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:03.043772 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:03.044636 (+ 864us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:43:03.045279 (+ 643us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:43:03.045497 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:03.045818 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 14:43:03.045866 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260810 14:43:03.057126 20015 raft_consensus.cc:1261] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15 [term 1 FOLLOWER]: Refusing update from remote peer 4e16dce3be2f4f598ee87e8da5c2d0cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:43:03.057109 20222 raft_consensus.cc:1261] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc [term 1 FOLLOWER]: Refusing update from remote peer 4e16dce3be2f4f598ee87e8da5c2d0cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:43:03.058282 20526 consensus_queue.cc:1035] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "232de782c610445c819164b562a9eacc" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 39657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:43:03.058748 20516 consensus_queue.cc:1035] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "37f0413553c447628ce8d71f826a4f15" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 35387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:43:03.076304 20531 mvcc.cc:205] Tried to move back new op lower bound from 7316983738588192768 to 7316983738356699136. Current Snapshot: MvccSnapshot[applied={T|T < 7316983738588192768}]
I20260810 14:43:03.080721 20533 mvcc.cc:205] Tried to move back new op lower bound from 7316983738588192768 to 7316983738356699136. Current Snapshot: MvccSnapshot[applied={T|T < 7316983738588192768}]
I20260810 14:43:03.089468 19793 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:43:03.089897 19793 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:43:03.094791 19793 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45d114332fff407fafe3dec8b2d30ea1: Sending DeleteTablet for 3 replicas of tablet be2e654e699e4574844e187a173da8c3
I20260810 14:43:03.096448 19987 tablet_service.cc:1514] Processing DeleteTablet for tablet be2e654e699e4574844e187a173da8c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:43:03 UTC) from {username='slave'} at 127.0.0.1:54658
I20260810 14:43:03.096539 20396 tablet_service.cc:1514] Processing DeleteTablet for tablet be2e654e699e4574844e187a173da8c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:43:03 UTC) from {username='slave'} at 127.0.0.1:38274
I20260810 14:43:03.097090 20185 tablet_service.cc:1514] Processing DeleteTablet for tablet be2e654e699e4574844e187a173da8c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:43:03 UTC) from {username='slave'} at 127.0.0.1:39404
I20260810 14:43:03.099952 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 19896
I20260810 14:43:03.109094 20548 tablet_replica.cc:330] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: stopping tablet replica
I20260810 14:43:03.109194 20551 tablet_replica.cc:330] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: stopping tablet replica
I20260810 14:43:03.109724 20551 raft_consensus.cc:2229] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:43:03.110412 20551 raft_consensus.cc:2258] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:43:03.110389 20548 raft_consensus.cc:2229] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:43:03.111423 20548 raft_consensus.cc:2258] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:43:03.114684 20548 ts_tablet_manager.cc:1905] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:43:03.123920 20548 ts_tablet_manager.cc:1918] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:43:03.124209 20548 log.cc:1196] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/be2e654e699e4574844e187a173da8c3
I20260810 14:43:03.124917 20548 ts_tablet_manager.cc:1939] T be2e654e699e4574844e187a173da8c3 P 4e16dce3be2f4f598ee87e8da5c2d0cc: Deleting consensus metadata
I20260810 14:43:03.125095 20551 ts_tablet_manager.cc:1905] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:43:03.127031 19775 catalog_manager.cc:4841] TS 4e16dce3be2f4f598ee87e8da5c2d0cc (127.21.55.67:36979): tablet be2e654e699e4574844e187a173da8c3 (table test-table [id=3270c30d6c8641078ccb3a4bd141bc35]) successfully deleted
I20260810 14:43:03.127907 20554 tablet_replica.cc:330] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: stopping tablet replica
I20260810 14:43:03.128417 20554 raft_consensus.cc:2229] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:43:03.128957 20554 raft_consensus.cc:2258] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:43:03.131270 20551 ts_tablet_manager.cc:1918] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:43:03.131458 20551 log.cc:1196] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/be2e654e699e4574844e187a173da8c3
I20260810 14:43:03.131942 20551 ts_tablet_manager.cc:1939] T be2e654e699e4574844e187a173da8c3 P 37f0413553c447628ce8d71f826a4f15: Deleting consensus metadata
I20260810 14:43:03.134841 19778 catalog_manager.cc:4841] TS 37f0413553c447628ce8d71f826a4f15 (127.21.55.65:35387): tablet be2e654e699e4574844e187a173da8c3 (table test-table [id=3270c30d6c8641078ccb3a4bd141bc35]) successfully deleted
I20260810 14:43:03.137286 20554 ts_tablet_manager.cc:1905] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:43:03.139204 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:03.102056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54666 (local address 127.21.55.65:35387)
0810 14:43:03.102345 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:03.102356 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:03.123081 (+ 20725us) server_negotiation.cc:407] Connection header received
0810 14:43:03.123395 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:03.123402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:03.123515 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:03.123600 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:03.124358 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:03.125501 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:03.134852 (+ 9351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:03.135679 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:03.135831 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:03.137447 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:03.137458 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:03.137476 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:03.138232 (+ 756us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:03.138287 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:03.138312 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:03.138402 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:03.138703 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 14:43:03.138825 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 14:43:03.140697 20081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:03.143575 20554 ts_tablet_manager.cc:1918] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:43:03.143747 20554 log.cc:1196] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/be2e654e699e4574844e187a173da8c3
I20260810 14:43:03.144309 20554 ts_tablet_manager.cc:1939] T be2e654e699e4574844e187a173da8c3 P 232de782c610445c819164b562a9eacc: Deleting consensus metadata
I20260810 14:43:03.146303 19777 catalog_manager.cc:4841] TS 232de782c610445c819164b562a9eacc (127.21.55.66:39657): tablet be2e654e699e4574844e187a173da8c3 (table test-table [id=3270c30d6c8641078ccb3a4bd141bc35]) successfully deleted
I20260810 14:43:03.246608 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 19896
I20260810 14:43:03.281904 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 20093
I20260810 14:43:03.290335 20506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:03.283060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39406 (local address 127.21.55.66:39657)
0810 14:43:03.283345 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:03.283353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:03.283379 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:43:03.283455 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:03.283461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:03.283590 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:03.283720 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:03.284181 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:03.285366 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:03.286765 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:03.287431 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:03.287570 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:03.288659 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:03.288670 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:03.288689 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:03.289480 (+ 791us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:03.289558 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:03.289583 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:03.289698 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:03.289840 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 14:43:03.289952 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260810 14:43:03.291174 20284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:03.435766 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 20093
I20260810 14:43:03.463727 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 20298
I20260810 14:43:03.471334 20501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:03.464550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38294 (local address 127.21.55.67:36979)
0810 14:43:03.464729 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:03.464737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:03.464761 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:43:03.464990 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:03.464996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:03.465103 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:03.465214 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:03.465637 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:03.466482 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:03.467848 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:03.468337 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:03.468499 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:03.469661 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:03.469670 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:03.469684 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:03.470427 (+ 743us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:03.470501 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:03.470579 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:03.470688 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:03.470846 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 14:43:03.470942 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:43:03.472168 20476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:03.598330 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 20298
I20260810 14:43:03.631392 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 19753
I20260810 14:43:03.632498 19842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:03.743520 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 19753
2026-08-10T14:43:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3179 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:03 dist-test-slave-h6n0 krb5kdc[20569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:03 dist-test-slave-h6n0 krb5kdc[20569](info): set up 2 sockets
Aug 10 14:43:03 dist-test-slave-h6n0 krb5kdc[20569](info): commencing operation
krb5kdc: starting...
W20260810 14:43:05.118930 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.006s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:05 dist-test-slave-h6n0 krb5kdc[20569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:43:05.429862 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:40529
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:40529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:05.779088 20591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:05.779353 20591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:05.779521 20591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:05.786698 20591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:05.786800 20591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:05.786900 20591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:05.787006 20591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:05.787086 20591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:05.798247 20591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:40529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:40529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:05.799638 20591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:05.800951 20591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:05.815310 20603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:05.815313 20601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:05.815739 20599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:05.816367 20591 server_base.cc:1034] running on GCE node
I20260810 14:43:05.817322 20591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:05.819129 20591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:05.820438 20591 hybrid_clock.cc:648] HybridClock initialized: now 1786372985820372 us; error 38 us; skew 500 ppm
I20260810 14:43:05.820873 20591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:05 dist-test-slave-h6n0 krb5kdc[20569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372985, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:05.828327 20591 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:43:05.830132 20591 webserver.cc:460] Webserver started at http://127.21.55.126:45717/ using document root <none> and password file <none>
I20260810 14:43:05.830919 20591 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:05.831072 20591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:05.831410 20591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:05.834649 20591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "641f59be4369491eb52e26b9154d645a"
format_stamp: "Formatted at 2026-08-10 14:43:05 on dist-test-slave-h6n0"
server_key: "b8bdbbf0e745bfaf42b2a569dba074d1"
server_key_iv: "b3b1ffc92d611f338d19227653da3a8e"
server_key_version: "encryptionkey@0"
I20260810 14:43:05.836006 20591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "641f59be4369491eb52e26b9154d645a"
format_stamp: "Formatted at 2026-08-10 14:43:05 on dist-test-slave-h6n0"
server_key: "b8bdbbf0e745bfaf42b2a569dba074d1"
server_key_iv: "b3b1ffc92d611f338d19227653da3a8e"
server_key_version: "encryptionkey@0"
I20260810 14:43:05.843374 20591 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 14:43:05.848240 20612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:05.850004 20591 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260810 14:43:05.850215 20591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "641f59be4369491eb52e26b9154d645a"
format_stamp: "Formatted at 2026-08-10 14:43:05 on dist-test-slave-h6n0"
server_key: "b8bdbbf0e745bfaf42b2a569dba074d1"
server_key_iv: "b3b1ffc92d611f338d19227653da3a8e"
server_key_version: "encryptionkey@0"
I20260810 14:43:05.850468 20591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:05.872571 20591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:05.877192 20591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:05.877513 20591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:05.901433 20591 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:40529
I20260810 14:43:05.901530 20686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:40529 every 8 connection(s)
I20260810 14:43:05.902942 20591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:05.906582 20688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:05.910844 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 20591
I20260810 14:43:05.911053 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:05.911615 21725 external_mini_cluster.cc:1428] Setting key 929791dacd6f958568988f43f18a5efb
I20260810 14:43:05.917847 20688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a: Bootstrap starting.
I20260810 14:43:05.921061 20688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:05.922051 20688 log.cc:824] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:05.925436 20688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a: No bootstrap required, opened a new log
Aug 10 14:43:05 dist-test-slave-h6n0 krb5kdc[20569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:43:05.935163 20688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641f59be4369491eb52e26b9154d645a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40529 } }
I20260810 14:43:05.935595 20688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:05.935720 20688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641f59be4369491eb52e26b9154d645a, State: Initialized, Role: FOLLOWER
I20260810 14:43:05.936345 20688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641f59be4369491eb52e26b9154d645a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40529 } }
I20260810 14:43:05.936611 20688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:05.936786 20688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:05.936987 20688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:05.938212 20692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:05.913579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.21.55.126:40529)
0810 14:43:05.914360 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:05.914642 (+ 282us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:05.914785 (+ 143us) server_negotiation.cc:407] Connection header received
0810 14:43:05.916859 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:05.916882 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:05.917253 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:05.917648 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:05.919250 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:05.920638 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:05.922081 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:05.923139 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:05.923351 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:05.929169 (+ 5818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:05.929180 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:05.929201 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:05.929257 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:05.933646 (+ 4389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:05.934429 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:05.934438 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:05.934446 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:05.934697 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:05.935110 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:05.935116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:05.935119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:05.935846 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:05.936142 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:05.936621 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 14:43:05.937214 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":384,"threads_started":1}
I20260810 14:43:05.939455 20688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641f59be4369491eb52e26b9154d645a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40529 } }
I20260810 14:43:05.939934 20688 leader_election.cc:302] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 641f59be4369491eb52e26b9154d645a; no voters:
I20260810 14:43:05.940563 20688 leader_election.cc:288] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:05.940701 20694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:05.941913 20694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [term 1 LEADER]: Becoming Leader. State: Replica: 641f59be4369491eb52e26b9154d645a, State: Running, Role: LEADER
I20260810 14:43:05.942387 20694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641f59be4369491eb52e26b9154d645a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40529 } }
I20260810 14:43:05.942631 20688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:05.945443 20695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "641f59be4369491eb52e26b9154d645a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641f59be4369491eb52e26b9154d645a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40529 } } }
I20260810 14:43:05.945429 20698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 641f59be4369491eb52e26b9154d645a. Latest consensus state: current_term: 1 leader_uuid: "641f59be4369491eb52e26b9154d645a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "641f59be4369491eb52e26b9154d645a" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 40529 } } }
I20260810 14:43:05.946219 20695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:05.946230 20698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:05.947890 20707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:05.953964 20707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:05.963390 20707 catalog_manager.cc:1347] Generated new cluster ID: 1f4497f99d324d6aa8898367e6ee2ee7
I20260810 14:43:05.963534 20707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:05.981395 20707 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:05.983246 20707 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:05.992362 20707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a: Generated new TSK 0
I20260810 14:43:05.993335 20707 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.55.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
Entry for principal kudu/127.21.55.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab.
I20260810 14:43:06.082667 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40529
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:06.473845 20739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:06.474121 20739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:06.474216 20739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:06.481400 20739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:06.481513 20739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:06.481627 20739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:43:06.491317 20739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.65.keytab
--principal=kudu/127.21.55.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:06.492672 20739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:06.494068 20739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:06.510260 20752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:06.510416 20753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:06.511262 20755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:06.511833 20739 server_base.cc:1034] running on GCE node
I20260810 14:43:06.512537 20739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:06.513676 20739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:06.515005 20739 hybrid_clock.cc:648] HybridClock initialized: now 1786372986514962 us; error 42 us; skew 500 ppm
I20260810 14:43:06.515362 20739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:06 dist-test-slave-h6n0 krb5kdc[20569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372986, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:06.519420 20739 init.cc:377] Logged in from keytab as kudu/127.21.55.65@KRBTEST.COM (short username kudu)
I20260810 14:43:06.521050 20739 webserver.cc:460] Webserver started at http://127.21.55.65:35223/ using document root <none> and password file <none>
I20260810 14:43:06.521747 20739 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:06.521855 20739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:06.522182 20739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:06.525257 20739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "77580681b0444d4bbd1f5ec7ce1dd102"
format_stamp: "Formatted at 2026-08-10 14:43:06 on dist-test-slave-h6n0"
server_key: "717a56acc4f94735030ba8c5c5d6301c"
server_key_iv: "2e8d3ac20b10172e5e3f052ac91c7387"
server_key_version: "encryptionkey@0"
I20260810 14:43:06.526038 20739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "77580681b0444d4bbd1f5ec7ce1dd102"
format_stamp: "Formatted at 2026-08-10 14:43:06 on dist-test-slave-h6n0"
server_key: "717a56acc4f94735030ba8c5c5d6301c"
server_key_iv: "2e8d3ac20b10172e5e3f052ac91c7387"
server_key_version: "encryptionkey@0"
I20260810 14:43:06.531746 20739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260810 14:43:06.535491 20764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:06.536809 20739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:43:06.537014 20739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "77580681b0444d4bbd1f5ec7ce1dd102"
format_stamp: "Formatted at 2026-08-10 14:43:06 on dist-test-slave-h6n0"
server_key: "717a56acc4f94735030ba8c5c5d6301c"
server_key_iv: "2e8d3ac20b10172e5e3f052ac91c7387"
server_key_version: "encryptionkey@0"
I20260810 14:43:06.537207 20739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:06.553153 20739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:06.557889 20739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:06.558147 20739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:06.559342 20739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:43:06.560732 20739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:43:06.560809 20739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:06.560894 20739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:43:06.560951 20739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:06.596961 20739 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:42067
I20260810 14:43:06.597056 20934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:42067 every 8 connection(s)
I20260810 14:43:06.598299 20739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:43:06.602681 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 20739
I20260810 14:43:06.602880 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:43:06.603386 21725 external_mini_cluster.cc:1428] Setting key 5b507c86eed36d1f292182efeffc1a36
Aug 10 14:43:06 dist-test-slave-h6n0 krb5kdc[20569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372986, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.65@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:43:06.622934 20938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:06.601030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:34413 (local address 127.21.55.126:40529)
0810 14:43:06.601860 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:06.601868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:06.603479 (+ 1611us) server_negotiation.cc:407] Connection header received
0810 14:43:06.606443 (+ 2964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:06.606450 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:06.606586 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:06.606727 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:06.608337 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:06.609524 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:06.611299 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:06.611990 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:06.612150 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:06.615967 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:06.615976 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:06.615984 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:06.616038 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:06.619445 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:06.620453 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:06.620457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:06.620460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:06.620589 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:06.621107 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:06.621111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:06.621113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:06.621401 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:06.621518 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:06.622399 (+ 881us) server_negotiation.cc:299] Negotiation successful
0810 14:43:06.622645 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":382,"threads_started":1}
I20260810 14:43:06.624476 20940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:06.602100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:40529 (local address 127.21.55.65:34413)
0810 14:43:06.603213 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:06.603279 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:06.606085 (+ 2806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:06.606964 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:06.606972 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:06.607540 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:06.608148 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:06.608166 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:06.609734 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:06.609740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:06.611137 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:06.611151 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:06.612268 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:06.612274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:06.612478 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:06.613240 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:06.613262 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:06.615747 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:06.619588 (+ 3841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:06.619600 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:06.619616 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:06.620294 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:06.620707 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:06.620715 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:06.620721 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:06.620962 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:06.621532 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:06.621543 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:06.621989 (+ 446us) client_negotiation.cc:769] Sending connection context
0810 14:43:06.622284 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 14:43:06.622969 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":500,"threads_started":1}
I20260810 14:43:06.626657 20935 heartbeater.cc:347] Connected to a master server at 127.21.55.126:40529
I20260810 14:43:06.626991 20935 heartbeater.cc:464] Registering TS with master...
I20260810 14:43:06.627751 20935 heartbeater.cc:511] Master 127.21.55.126:40529 requested a full tablet report, sending...
I20260810 14:43:06.629832 20634 ts_manager.cc:194] Registered new tserver with Master: 77580681b0444d4bbd1f5ec7ce1dd102 (127.21.55.65:42067)
I20260810 14:43:06.631410 20634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.65@KRBTEST.COM'} at 127.21.55.65:34413
WARNING: no policy specified for kudu/127.21.55.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
Entry for principal kudu/127.21.55.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab.
I20260810 14:43:06.674546 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40529
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:07.043720 20946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:07.044143 20946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:07.044332 20946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:07.060323 20946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:07.060503 20946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:07.060818 20946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:43:07.075273 20946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.66.keytab
--principal=kudu/127.21.55.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:07.076557 20946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:07.077781 20946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:07.091372 20951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:07.091492 20952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:07.092216 20955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:07.093220 20946 server_base.cc:1034] running on GCE node
I20260810 14:43:07.093961 20946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:07.095083 20946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:07.096349 20946 hybrid_clock.cc:648] HybridClock initialized: now 1786372987096288 us; error 56 us; skew 500 ppm
I20260810 14:43:07.096693 20946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:07 dist-test-slave-h6n0 krb5kdc[20569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372987, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:07.100883 20946 init.cc:377] Logged in from keytab as kudu/127.21.55.66@KRBTEST.COM (short username kudu)
I20260810 14:43:07.102896 20946 webserver.cc:460] Webserver started at http://127.21.55.66:42121/ using document root <none> and password file <none>
I20260810 14:43:07.103942 20946 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:07.104085 20946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:07.104431 20946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:07.108247 20946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "94508f3a18584de4b014049780f0de85"
format_stamp: "Formatted at 2026-08-10 14:43:07 on dist-test-slave-h6n0"
server_key: "a3bfece29b18601b4d06b0710712870c"
server_key_iv: "26575bdf82b01b8a09c2fcd0b1a14d1e"
server_key_version: "encryptionkey@0"
I20260810 14:43:07.109354 20946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "94508f3a18584de4b014049780f0de85"
format_stamp: "Formatted at 2026-08-10 14:43:07 on dist-test-slave-h6n0"
server_key: "a3bfece29b18601b4d06b0710712870c"
server_key_iv: "26575bdf82b01b8a09c2fcd0b1a14d1e"
server_key_version: "encryptionkey@0"
I20260810 14:43:07.115818 20946 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 14:43:07.119437 20963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:07.120951 20946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:43:07.121094 20946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "94508f3a18584de4b014049780f0de85"
format_stamp: "Formatted at 2026-08-10 14:43:07 on dist-test-slave-h6n0"
server_key: "a3bfece29b18601b4d06b0710712870c"
server_key_iv: "26575bdf82b01b8a09c2fcd0b1a14d1e"
server_key_version: "encryptionkey@0"
I20260810 14:43:07.121349 20946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:07.152855 20946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:07.158216 20946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:07.158586 20946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:07.159693 20946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:43:07.161134 20946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:43:07.161211 20946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:07.161352 20946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:43:07.161406 20946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:07.200907 20946 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:44803
I20260810 14:43:07.200997 21139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:44803 every 8 connection(s)
I20260810 14:43:07.202219 20946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:43:07.210314 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 20946
I20260810 14:43:07.210563 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:43:07.211170 21725 external_mini_cluster.cc:1428] Setting key 8995c6c8b1324a31672c9a5b2d38ad26
Aug 10 14:43:07 dist-test-slave-h6n0 krb5kdc[20569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372987, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.66@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:43:07.226588 21149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:07.205257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:33009 (local address 127.21.55.126:40529)
0810 14:43:07.205939 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:07.205947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:07.207373 (+ 1426us) server_negotiation.cc:407] Connection header received
0810 14:43:07.209272 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:07.209277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:07.209361 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:07.209468 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:07.211128 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.211904 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.214033 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.214495 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.214704 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:07.219080 (+ 4376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:07.219086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:07.219090 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:07.219124 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:07.223287 (+ 4163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:07.224047 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:07.224054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:07.224062 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:07.224180 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:07.224655 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:07.224658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:07.224661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:07.224982 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:07.225158 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:07.225948 (+ 790us) server_negotiation.cc:299] Negotiation successful
0810 14:43:07.226170 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":319,"threads_started":1}
I20260810 14:43:07.227617 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:07.206188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:40529 (local address 127.21.55.66:33009)
0810 14:43:07.207151 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:07.207199 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:07.208967 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:07.209685 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:07.209692 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:07.210235 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:07.210950 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:07.210967 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.212287 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.212297 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:07.213866 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:07.213880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.214812 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.214821 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:07.215318 (+ 497us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:07.216329 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:07.216361 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:07.218753 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:07.223446 (+ 4693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:07.223457 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:07.223476 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:07.223895 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:07.224299 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:07.224305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:07.224313 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:07.224500 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:07.225146 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:07.225155 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:07.225492 (+ 337us) client_negotiation.cc:769] Sending connection context
0810 14:43:07.225968 (+ 476us) client_negotiation.cc:240] Negotiation successful
0810 14:43:07.226487 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":430,"threads_started":1}
I20260810 14:43:07.229166 21140 heartbeater.cc:347] Connected to a master server at 127.21.55.126:40529
I20260810 14:43:07.229511 21140 heartbeater.cc:464] Registering TS with master...
I20260810 14:43:07.230250 21140 heartbeater.cc:511] Master 127.21.55.126:40529 requested a full tablet report, sending...
I20260810 14:43:07.231740 20634 ts_manager.cc:194] Registered new tserver with Master: 94508f3a18584de4b014049780f0de85 (127.21.55.66:44803)
I20260810 14:43:07.232553 20634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.66@KRBTEST.COM'} at 127.21.55.66:33009
WARNING: no policy specified for kudu/127.21.55.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
Entry for principal kudu/127.21.55.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab.
I20260810 14:43:07.296271 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40529
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:43:07.635191 20935 heartbeater.cc:503] Master 127.21.55.126:40529 was elected leader, sending a full tablet report...
W20260810 14:43:07.672596 21156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:07.673090 21156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:07.673275 21156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:07.681306 21156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:07.681443 21156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:07.681577 21156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:43:07.692008 21156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.67.keytab
--principal=kudu/127.21.55.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:40529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:07.693187 21156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:07.694586 21156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:07.707769 21164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:07.707798 21165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:07.708432 21167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:07.708778 21156 server_base.cc:1034] running on GCE node
I20260810 14:43:07.709437 21156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:07.710439 21156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:07.711828 21156 hybrid_clock.cc:648] HybridClock initialized: now 1786372987711714 us; error 53 us; skew 500 ppm
I20260810 14:43:07.712543 21156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:07 dist-test-slave-h6n0 krb5kdc[20569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372987, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:07.716507 21156 init.cc:377] Logged in from keytab as kudu/127.21.55.67@KRBTEST.COM (short username kudu)
I20260810 14:43:07.718622 21156 webserver.cc:460] Webserver started at http://127.21.55.67:42579/ using document root <none> and password file <none>
I20260810 14:43:07.719694 21156 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:07.719889 21156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:07.720243 21156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:07.723305 21156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "072751a8686947b18e58094d7dcad851"
format_stamp: "Formatted at 2026-08-10 14:43:07 on dist-test-slave-h6n0"
server_key: "8a7885a25e6df5f52e76bbe08a5bd3be"
server_key_iv: "33b27e03a8af7b5eff95d537ffe4f35a"
server_key_version: "encryptionkey@0"
I20260810 14:43:07.724097 21156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "072751a8686947b18e58094d7dcad851"
format_stamp: "Formatted at 2026-08-10 14:43:07 on dist-test-slave-h6n0"
server_key: "8a7885a25e6df5f52e76bbe08a5bd3be"
server_key_iv: "33b27e03a8af7b5eff95d537ffe4f35a"
server_key_version: "encryptionkey@0"
I20260810 14:43:07.729611 21156 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260810 14:43:07.733129 21175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:07.734373 21156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:43:07.734666 21156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "072751a8686947b18e58094d7dcad851"
format_stamp: "Formatted at 2026-08-10 14:43:07 on dist-test-slave-h6n0"
server_key: "8a7885a25e6df5f52e76bbe08a5bd3be"
server_key_iv: "33b27e03a8af7b5eff95d537ffe4f35a"
server_key_version: "encryptionkey@0"
I20260810 14:43:07.734871 21156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:07.767318 21156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:07.771855 21156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:07.772166 21156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:07.773303 21156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:43:07.774734 21156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:43:07.774842 21156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:07.774996 21156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:43:07.775089 21156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:07.815292 21156 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:45335
I20260810 14:43:07.815344 21355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:45335 every 8 connection(s)
I20260810 14:43:07.817039 21156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:43:07.827088 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 21156
I20260810 14:43:07.827339 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:43:07.827925 21725 external_mini_cluster.cc:1428] Setting key a052af887447dfdf045c91caa071f994
Aug 10 14:43:07 dist-test-slave-h6n0 krb5kdc[20569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372987, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.67@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:43:07.844121 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:07.820359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:47355 (local address 127.21.55.126:40529)
0810 14:43:07.820966 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:07.820972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:07.822175 (+ 1203us) server_negotiation.cc:407] Connection header received
0810 14:43:07.824717 (+ 2542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:07.824724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:07.824860 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:07.824999 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:07.827728 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.829049 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.831403 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.832074 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.832346 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:07.837363 (+ 5017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:07.837369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:07.837383 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:07.837418 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:07.840583 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:07.841470 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:07.841476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:07.841480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:07.841618 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:07.842138 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:07.842142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:07.842144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:07.842426 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:07.842638 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:07.843435 (+ 797us) server_negotiation.cc:299] Negotiation successful
0810 14:43:07.843676 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":270,"threads_started":1}
I20260810 14:43:07.845144 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:07.820981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:40529 (local address 127.21.55.67:47355)
0810 14:43:07.821883 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:07.821953 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:07.824237 (+ 2284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:07.825339 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:07.825349 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:07.826351 (+ 1002us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:07.827438 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:07.827472 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.829401 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.829409 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:07.831208 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:07.831227 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.832416 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.832422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:07.832644 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:07.833607 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:07.833631 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:07.837020 (+ 3389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:07.840746 (+ 3726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:07.840758 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:07.840771 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:07.841331 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:07.841739 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:07.841746 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:07.841755 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:07.841967 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:07.842587 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:07.842595 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:07.842951 (+ 356us) client_negotiation.cc:769] Sending connection context
0810 14:43:07.843322 (+ 371us) client_negotiation.cc:240] Negotiation successful
0810 14:43:07.843855 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":383,"threads_started":1}
I20260810 14:43:07.846746 21358 heartbeater.cc:347] Connected to a master server at 127.21.55.126:40529
I20260810 14:43:07.847082 21358 heartbeater.cc:464] Registering TS with master...
I20260810 14:43:07.847836 21358 heartbeater.cc:511] Master 127.21.55.126:40529 requested a full tablet report, sending...
I20260810 14:43:07.849476 20634 ts_manager.cc:194] Registered new tserver with Master: 072751a8686947b18e58094d7dcad851 (127.21.55.67:45335)
I20260810 14:43:07.850443 20634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.55.67@KRBTEST.COM'} at 127.21.55.67:47355
I20260810 14:43:07.857213 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:43:07.876128 21360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:07.861619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34200 (local address 127.21.55.126:40529)
0810 14:43:07.861787 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:07.861810 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:07.862245 (+ 435us) server_negotiation.cc:407] Connection header received
0810 14:43:07.862453 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:07.862458 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:07.862523 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:07.862682 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:07.864346 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.865070 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.866369 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.866848 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.866988 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:07.870329 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:07.870336 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:07.870339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:07.870372 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:07.873236 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:07.873996 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:07.874001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:07.874003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:07.874122 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:07.874560 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:07.874567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:07.874572 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:07.874913 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:07.875190 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:07.875562 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 14:43:07.875789 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 14:43:07.881320 20634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:43:07.890090 20634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:43:07.932763 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:07.908420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:45335 (local address 127.0.0.1:49802)
0810 14:43:07.909312 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:07.909362 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:07.909599 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:07.923990 (+ 14391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:07.924001 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:07.924050 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:07.924663 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:07.924681 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.927124 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.927133 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:07.929378 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:07.929391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.931636 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.931643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:07.931867 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:07.931937 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 14:43:07.932059 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 14:43:07.932272 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":346,"threads_started":1}
I20260810 14:43:07.933678 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:07.908596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49802 (local address 127.21.55.67:45335)
0810 14:43:07.923136 (+ 14540us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:07.923150 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:07.923193 (+ 43us) server_negotiation.cc:407] Connection header received
0810 14:43:07.923324 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:07.923335 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:07.923671 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:07.923920 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:07.924979 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.926889 (+ 1910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.929557 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.931411 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.932823 (+ 1412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:07.933094 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:07.933175 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 14:43:07.933337 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14305,"thread_start_us":14118,"threads_started":1}
I20260810 14:43:07.937009 21257 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0b6d755f0d42a7a58415a8b954e82a (DEFAULT_TABLE table=test-table [id=f2b6ff971e694857b81210f22c42c5ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:43:07.938735 21257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0b6d755f0d42a7a58415a8b954e82a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:07.940201 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:07.910013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:44803 (local address 127.0.0.1:39102)
0810 14:43:07.930557 (+ 20544us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:07.930577 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:07.930702 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:07.932133 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:07.932137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:07.932166 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:07.932548 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:07.932559 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.934456 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.934462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:07.937026 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:07.937042 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.938786 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.938792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:07.938960 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:07.939137 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 14:43:07.939347 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 14:43:07.939563 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20390,"mutex_wait_us":104,"spinlock_wait_cycles":3328,"thread_start_us":20900,"threads_started":1}
I20260810 14:43:07.940374 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:07.910745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.21.55.66:44803)
0810 14:43:07.928427 (+ 17682us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:07.928445 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:07.931262 (+ 2817us) server_negotiation.cc:407] Connection header received
0810 14:43:07.931385 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:07.931393 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:07.931643 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:07.931857 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:07.932729 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.934296 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.937221 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.938245 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.938388 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:07.938635 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:07.939861 (+ 1226us) server_negotiation.cc:299] Negotiation successful
0810 14:43:07.940033 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17486,"thread_start_us":17678,"threads_started":1}
I20260810 14:43:07.940958 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:07.914335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40146 (local address 127.21.55.65:42067)
0810 14:43:07.930080 (+ 15745us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:07.930089 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:07.931333 (+ 1244us) server_negotiation.cc:407] Connection header received
0810 14:43:07.931629 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:07.931637 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:07.931846 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:07.932019 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:07.933043 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.934433 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.937212 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.939016 (+ 1804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.939169 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:07.939379 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:07.940600 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0810 14:43:07.940722 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15555,"thread_start_us":15397,"threads_started":1}
I20260810 14:43:07.941949 21375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:07.913849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42067 (local address 127.0.0.1:40146)
0810 14:43:07.931224 (+ 17375us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:07.931241 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:07.931435 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:07.932289 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:07.932293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:07.932322 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:07.932824 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:07.932838 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.934651 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.934656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:07.937026 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:07.937042 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:07.939598 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:07.939607 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:07.940320 (+ 713us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:07.940365 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 14:43:07.941388 (+ 1023us) client_negotiation.cc:240] Negotiation successful
0810 14:43:07.941544 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17262,"mutex_wait_us":78,"thread_start_us":16389,"threads_started":1}
I20260810 14:43:07.944247 21041 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0b6d755f0d42a7a58415a8b954e82a (DEFAULT_TABLE table=test-table [id=f2b6ff971e694857b81210f22c42c5ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:43:07.945674 21041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0b6d755f0d42a7a58415a8b954e82a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:07.945421 20832 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0b6d755f0d42a7a58415a8b954e82a (DEFAULT_TABLE table=test-table [id=f2b6ff971e694857b81210f22c42c5ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:43:07.946694 20832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0b6d755f0d42a7a58415a8b954e82a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:07.951581 21381 tablet_bootstrap.cc:492] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: Bootstrap starting.
I20260810 14:43:07.954603 21381 tablet_bootstrap.cc:654] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:07.955624 21381 log.cc:824] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:07.958170 21381 tablet_bootstrap.cc:492] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: No bootstrap required, opened a new log
I20260810 14:43:07.958518 21381 ts_tablet_manager.cc:1397] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260810 14:43:07.960903 21383 tablet_bootstrap.cc:492] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85: Bootstrap starting.
I20260810 14:43:07.964351 21383 tablet_bootstrap.cc:654] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:07.965160 21386 tablet_bootstrap.cc:492] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102: Bootstrap starting.
I20260810 14:43:07.965583 21383 log.cc:824] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:07.968266 21386 tablet_bootstrap.cc:654] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:07.968724 21383 tablet_bootstrap.cc:492] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85: No bootstrap required, opened a new log
I20260810 14:43:07.969000 21383 ts_tablet_manager.cc:1397] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 14:43:07.969427 21386 log.cc:824] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:07.969379 21381 raft_consensus.cc:348] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072751a8686947b18e58094d7dcad851" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45335 } } peers { permanent_uuid: "94508f3a18584de4b014049780f0de85" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 44803 } } peers { permanent_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42067 } }
I20260810 14:43:07.969816 21381 raft_consensus.cc:374] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:07.969980 21381 raft_consensus.cc:729] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 072751a8686947b18e58094d7dcad851, State: Initialized, Role: FOLLOWER
I20260810 14:43:07.970749 21381 consensus_queue.cc:260] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072751a8686947b18e58094d7dcad851" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45335 } } peers { permanent_uuid: "94508f3a18584de4b014049780f0de85" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 44803 } } peers { permanent_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42067 } }
I20260810 14:43:07.972357 21386 tablet_bootstrap.cc:492] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102: No bootstrap required, opened a new log
I20260810 14:43:07.972635 21386 ts_tablet_manager.cc:1397] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 14:43:07.977975 21358 heartbeater.cc:503] Master 127.21.55.126:40529 was elected leader, sending a full tablet report...
I20260810 14:43:07.978961 21381 ts_tablet_manager.cc:1428] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: Time spent starting tablet: real 0.020s user 0.016s sys 0.003s
I20260810 14:43:07.980265 21386 raft_consensus.cc:348] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072751a8686947b18e58094d7dcad851" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45335 } } peers { permanent_uuid: "94508f3a18584de4b014049780f0de85" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 44803 } } peers { permanent_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42067 } }
I20260810 14:43:07.980690 21386 raft_consensus.cc:374] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:07.980790 21386 raft_consensus.cc:729] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77580681b0444d4bbd1f5ec7ce1dd102, State: Initialized, Role: FOLLOWER
I20260810 14:43:07.980883 21383 raft_consensus.cc:348] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072751a8686947b18e58094d7dcad851" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45335 } } peers { permanent_uuid: "94508f3a18584de4b014049780f0de85" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 44803 } } peers { permanent_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42067 } }
I20260810 14:43:07.981290 21383 raft_consensus.cc:374] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:07.981411 21383 raft_consensus.cc:729] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94508f3a18584de4b014049780f0de85, State: Initialized, Role: FOLLOWER
I20260810 14:43:07.981407 21386 consensus_queue.cc:260] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072751a8686947b18e58094d7dcad851" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45335 } } peers { permanent_uuid: "94508f3a18584de4b014049780f0de85" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 44803 } } peers { permanent_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42067 } }
I20260810 14:43:07.981995 21383 consensus_queue.cc:260] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072751a8686947b18e58094d7dcad851" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45335 } } peers { permanent_uuid: "94508f3a18584de4b014049780f0de85" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 44803 } } peers { permanent_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42067 } }
I20260810 14:43:07.983279 21386 ts_tablet_manager.cc:1428] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102: Time spent starting tablet: real 0.011s user 0.006s sys 0.006s
I20260810 14:43:07.984386 21140 heartbeater.cc:503] Master 127.21.55.126:40529 was elected leader, sending a full tablet report...
I20260810 14:43:07.985033 21383 ts_tablet_manager.cc:1428] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260810 14:43:08.002399 21391 raft_consensus.cc:482] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:43:08.002856 21391 raft_consensus.cc:504] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072751a8686947b18e58094d7dcad851" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45335 } } peers { permanent_uuid: "94508f3a18584de4b014049780f0de85" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 44803 } } peers { permanent_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42067 } }
I20260810 14:43:08.004859 21391 leader_election.cc:288] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 072751a8686947b18e58094d7dcad851 (127.21.55.67:45335), 77580681b0444d4bbd1f5ec7ce1dd102 (127.21.55.65:42067)
I20260810 14:43:08.013670 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:08.005580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:54633 (local address 127.21.55.67:45335)
0810 14:43:08.005944 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:08.005952 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:08.006259 (+ 307us) server_negotiation.cc:407] Connection header received
0810 14:43:08.006500 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:08.006504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:08.006625 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:08.006727 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:08.007870 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.009263 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.011523 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.012754 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.012870 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:08.012973 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:08.013373 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 14:43:08.013475 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260810 14:43:08.013875 21392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:08.005392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:45335 (local address 127.21.55.66:54633)
0810 14:43:08.006105 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:08.006130 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:08.006332 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:08.006756 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:08.006760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:08.006788 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:08.007646 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:08.007662 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.009417 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.009424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:08.011344 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:08.011357 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.012990 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.012997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:08.013184 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:08.013219 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 14:43:08.013327 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 14:43:08.013462 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":3,"thread_start_us":386,"threads_started":1}
I20260810 14:43:08.015062 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:08.006897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:50391 (local address 127.21.55.65:42067)
0810 14:43:08.007222 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:08.007231 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:08.007885 (+ 654us) server_negotiation.cc:407] Connection header received
0810 14:43:08.008305 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:08.008317 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:08.008431 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:08.008548 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:08.009357 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.010500 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.012685 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.013956 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.014156 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:08.014223 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:08.014674 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 14:43:08.014805 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260810 14:43:08.015077 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:08.006422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42067 (local address 127.21.55.66:50391)
0810 14:43:08.007777 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:08.007802 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:08.007946 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:08.008735 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:08.008740 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:08.008765 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:08.009105 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:08.009116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.010701 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.010705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:08.012501 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:08.012516 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.014269 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.014278 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:08.014455 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:08.014477 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 14:43:08.014644 (+ 167us) client_negotiation.cc:240] Negotiation successful
0810 14:43:08.014768 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1214,"thread_start_us":977,"threads_started":1}
I20260810 14:43:08.015213 21284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0b6d755f0d42a7a58415a8b954e82a" candidate_uuid: "94508f3a18584de4b014049780f0de85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "072751a8686947b18e58094d7dcad851" is_pre_election: true
I20260810 14:43:08.015836 21284 raft_consensus.cc:2454] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94508f3a18584de4b014049780f0de85 in term 0.
I20260810 14:43:08.016101 20868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0b6d755f0d42a7a58415a8b954e82a" candidate_uuid: "94508f3a18584de4b014049780f0de85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" is_pre_election: true
I20260810 14:43:08.016587 20868 raft_consensus.cc:2454] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94508f3a18584de4b014049780f0de85 in term 0.
I20260810 14:43:08.016927 20969 leader_election.cc:302] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 072751a8686947b18e58094d7dcad851, 94508f3a18584de4b014049780f0de85; no voters:
I20260810 14:43:08.017350 21391 raft_consensus.cc:2781] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:43:08.017505 21391 raft_consensus.cc:482] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:43:08.017627 21391 raft_consensus.cc:3037] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:08.019932 21391 raft_consensus.cc:504] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072751a8686947b18e58094d7dcad851" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45335 } } peers { permanent_uuid: "94508f3a18584de4b014049780f0de85" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 44803 } } peers { permanent_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42067 } }
I20260810 14:43:08.020582 21391 leader_election.cc:288] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [CANDIDATE]: Term 1 election: Requested vote from peers 072751a8686947b18e58094d7dcad851 (127.21.55.67:45335), 77580681b0444d4bbd1f5ec7ce1dd102 (127.21.55.65:42067)
I20260810 14:43:08.021059 21284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0b6d755f0d42a7a58415a8b954e82a" candidate_uuid: "94508f3a18584de4b014049780f0de85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "072751a8686947b18e58094d7dcad851"
I20260810 14:43:08.021071 20868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0b6d755f0d42a7a58415a8b954e82a" candidate_uuid: "94508f3a18584de4b014049780f0de85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77580681b0444d4bbd1f5ec7ce1dd102"
I20260810 14:43:08.021275 21284 raft_consensus.cc:3037] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:08.021286 20868 raft_consensus.cc:3037] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:08.023566 20868 raft_consensus.cc:2454] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94508f3a18584de4b014049780f0de85 in term 1.
I20260810 14:43:08.023669 21284 raft_consensus.cc:2454] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94508f3a18584de4b014049780f0de85 in term 1.
I20260810 14:43:08.023993 20969 leader_election.cc:302] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77580681b0444d4bbd1f5ec7ce1dd102, 94508f3a18584de4b014049780f0de85; no voters:
I20260810 14:43:08.024284 21391 raft_consensus.cc:2781] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:08.024780 21391 raft_consensus.cc:686] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 1 LEADER]: Becoming Leader. State: Replica: 94508f3a18584de4b014049780f0de85, State: Running, Role: LEADER
I20260810 14:43:08.025251 21391 consensus_queue.cc:237] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072751a8686947b18e58094d7dcad851" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45335 } } peers { permanent_uuid: "94508f3a18584de4b014049780f0de85" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 44803 } } peers { permanent_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42067 } }
I20260810 14:43:08.031582 20634 catalog_manager.cc:5495] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94508f3a18584de4b014049780f0de85 (127.21.55.66). New cstate: current_term: 1 leader_uuid: "94508f3a18584de4b014049780f0de85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "072751a8686947b18e58094d7dcad851" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94508f3a18584de4b014049780f0de85" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 44803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42067 } health_report { overall_health: UNKNOWN } } }
W20260810 14:43:08.069114 21359 tablet.cc:2369] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:43:08.076800 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:08.071276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.21.55.66:44803)
0810 14:43:08.071464 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:08.071469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:08.071487 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:43:08.071654 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:08.071657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:08.071735 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:08.071865 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:43:08.072239 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.073004 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.074471 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.075130 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.075244 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:08.075699 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:43:08.076178 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:43:08.076291 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:08.076520 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 14:43:08.076586 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 14:43:08.085305 20868 raft_consensus.cc:1261] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102 [term 1 FOLLOWER]: Refusing update from remote peer 94508f3a18584de4b014049780f0de85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:43:08.085376 21284 raft_consensus.cc:1261] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851 [term 1 FOLLOWER]: Refusing update from remote peer 94508f3a18584de4b014049780f0de85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:43:08.086784 21395 consensus_queue.cc:1035] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77580681b0444d4bbd1f5ec7ce1dd102" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:43:08.087284 21391 consensus_queue.cc:1035] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "072751a8686947b18e58094d7dcad851" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 45335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:43:08.100258 21405 mvcc.cc:205] Tried to move back new op lower bound from 7316983759186931712 to 7316983758955417600. Current Snapshot: MvccSnapshot[applied={T|T < 7316983759186931712}]
I20260810 14:43:08.116495 20634 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:43:08.116724 20634 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:43:08.120596 20634 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 641f59be4369491eb52e26b9154d645a: Sending DeleteTablet for 3 replicas of tablet 3a0b6d755f0d42a7a58415a8b954e82a
I20260810 14:43:08.121803 21041 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0b6d755f0d42a7a58415a8b954e82a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:43:08 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:39102
I20260810 14:43:08.121840 21257 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0b6d755f0d42a7a58415a8b954e82a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:43:08 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:49802
I20260810 14:43:08.122047 20832 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0b6d755f0d42a7a58415a8b954e82a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:43:08 UTC) from {username='kudu', principal='kudu/127.21.55.126@KRBTEST.COM'} at 127.0.0.1:40146
I20260810 14:43:08.123237 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 20739
I20260810 14:43:08.134706 21419 tablet_replica.cc:330] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85: stopping tablet replica
I20260810 14:43:08.135468 21420 tablet_replica.cc:330] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102: stopping tablet replica
I20260810 14:43:08.136273 21420 raft_consensus.cc:2229] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:43:08.136984 21419 raft_consensus.cc:2229] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:43:08.137887 21420 raft_consensus.cc:2258] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:43:08.139938 21419 raft_consensus.cc:2258] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:43:08.143899 21418 tablet_replica.cc:330] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: stopping tablet replica
I20260810 14:43:08.144572 21418 raft_consensus.cc:2229] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:43:08.144920 21420 ts_tablet_manager.cc:1905] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:43:08.145484 21418 raft_consensus.cc:2258] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:43:08.145859 21419 ts_tablet_manager.cc:1905] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:43:08.148183 21418 ts_tablet_manager.cc:1905] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:43:08.155663 21420 ts_tablet_manager.cc:1918] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:43:08.155946 21420 log.cc:1196] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-0/wal/wals/3a0b6d755f0d42a7a58415a8b954e82a
I20260810 14:43:08.156603 21420 ts_tablet_manager.cc:1939] T 3a0b6d755f0d42a7a58415a8b954e82a P 77580681b0444d4bbd1f5ec7ce1dd102: Deleting consensus metadata
I20260810 14:43:08.157920 21418 ts_tablet_manager.cc:1918] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:43:08.158165 21418 log.cc:1196] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/3a0b6d755f0d42a7a58415a8b954e82a
I20260810 14:43:08.158805 21418 ts_tablet_manager.cc:1939] T 3a0b6d755f0d42a7a58415a8b954e82a P 072751a8686947b18e58094d7dcad851: Deleting consensus metadata
I20260810 14:43:08.158893 20618 catalog_manager.cc:4841] TS 77580681b0444d4bbd1f5ec7ce1dd102 (127.21.55.65:42067): tablet 3a0b6d755f0d42a7a58415a8b954e82a (table test-table [id=f2b6ff971e694857b81210f22c42c5ee]) successfully deleted
I20260810 14:43:08.160468 21419 ts_tablet_manager.cc:1918] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:43:08.160565 20618 catalog_manager.cc:4841] TS 072751a8686947b18e58094d7dcad851 (127.21.55.67:45335): tablet 3a0b6d755f0d42a7a58415a8b954e82a (table test-table [id=f2b6ff971e694857b81210f22c42c5ee]) successfully deleted
I20260810 14:43:08.160691 21419 log.cc:1196] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/3a0b6d755f0d42a7a58415a8b954e82a
I20260810 14:43:08.161262 21419 ts_tablet_manager.cc:1939] T 3a0b6d755f0d42a7a58415a8b954e82a P 94508f3a18584de4b014049780f0de85: Deleting consensus metadata
I20260810 14:43:08.163062 20615 catalog_manager.cc:4841] TS 94508f3a18584de4b014049780f0de85 (127.21.55.66:44803): tablet 3a0b6d755f0d42a7a58415a8b954e82a (table test-table [id=f2b6ff971e694857b81210f22c42c5ee]) successfully deleted
Aug 10 14:43:08 dist-test-slave-h6n0 krb5kdc[20569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.65@KRBTEST.COM
I20260810 14:43:08.174273 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:08.126032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40152 (local address 127.21.55.65:42067)
0810 14:43:08.126357 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:08.126366 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:08.143515 (+ 17149us) server_negotiation.cc:407] Connection header received
0810 14:43:08.151104 (+ 7589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:08.151112 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:08.151256 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:08.151349 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:08.158714 (+ 7365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.159853 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.161870 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.162488 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.162665 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:08.168271 (+ 5606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:08.168283 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:08.168297 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:08.168336 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:08.171321 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:08.172023 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:08.172030 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:08.172032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:08.172172 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:08.172606 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:08.172611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:08.172613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:08.172998 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:08.173318 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:08.173650 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 14:43:08.173912 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260810 14:43:08.175305 20930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:08.321089 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 20739
I20260810 14:43:08.355553 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 20946
Aug 10 14:43:08 dist-test-slave-h6n0 krb5kdc[20569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.66@KRBTEST.COM
I20260810 14:43:08.378579 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:08.356455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39128 (local address 127.21.55.66:44803)
0810 14:43:08.356663 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:08.356670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:08.356694 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:43:08.356823 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:08.356828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:08.356921 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:08.357036 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:08.358651 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.360025 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.361628 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.362302 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.362449 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:08.368857 (+ 6408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:08.368877 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:08.368899 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:08.368948 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:08.373949 (+ 5001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:08.375747 (+ 1798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:08.375758 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:08.375762 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:08.375986 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:08.376538 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:08.376544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:08.376548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:08.377055 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:08.377411 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:08.377709 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 14:43:08.377976 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:43:08.379491 21134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:08.519520 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 20946
I20260810 14:43:08.554935 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 21156
Aug 10 14:43:08 dist-test-slave-h6n0 krb5kdc[20569](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.67@KRBTEST.COM
I20260810 14:43:08.574348 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:08.556069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49812 (local address 127.21.55.67:45335)
0810 14:43:08.556610 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:08.556614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:08.556631 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:43:08.556690 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:08.556694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:08.556753 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:08.556868 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:08.558241 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.559398 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.560910 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:08.561498 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:08.561619 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:08.567281 (+ 5662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:08.567296 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:08.567312 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:08.567353 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:08.570736 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:08.571796 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:08.571804 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:08.571806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:08.571941 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:08.572773 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:08.572777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:08.572779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:08.573178 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:08.573412 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:08.573880 (+ 468us) server_negotiation.cc:299] Negotiation successful
0810 14:43:08.574064 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":313,"threads_started":1}
I20260810 14:43:08.575265 21348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:08.668570 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 21156
I20260810 14:43:08.693089 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 20591
I20260810 14:43:08.694443 20682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:08.811235 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 20591
2026-08-10T14:43:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5070 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T14:43:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:08Z Disabled control of system clock
I20260810 14:43:08.912420 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:46213
--webserver_interface=127.21.55.126
--webserver_port=0
--builtin_ntp_servers=127.21.55.84:34537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:46213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:43:09.279826 21433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:09.280225 21433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:09.280382 21433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:09.288605 21433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:09.288754 21433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:09.288833 21433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:09.288945 21433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:09.289023 21433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:09.300763 21433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:46213
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:46213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:09.302119 21433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:09.303995 21433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:09.319083 21439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:09.319378 21438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:09.319530 21442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:09.320401 21433 server_base.cc:1034] running on GCE node
I20260810 14:43:09.321214 21433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:09.322721 21433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:09.323993 21433 hybrid_clock.cc:648] HybridClock initialized: now 1786372989323933 us; error 41 us; skew 500 ppm
I20260810 14:43:09.324404 21433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:43:09.326881 21433 webserver.cc:460] Webserver started at http://127.21.55.126:36553/ using document root <none> and password file <none>
I20260810 14:43:09.327636 21433 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:09.327733 21433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:09.328097 21433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:09.331094 21433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "5bce0251e3674dea81f6ffb6b96467aa"
format_stamp: "Formatted at 2026-08-10 14:43:09 on dist-test-slave-h6n0"
server_key: "f5d6da2611e5f8ba6837db1e5ec77553"
server_key_iv: "89b11a2dd8fd1aab30a995a8ca151812"
server_key_version: "encryptionkey@0"
I20260810 14:43:09.331992 21433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "5bce0251e3674dea81f6ffb6b96467aa"
format_stamp: "Formatted at 2026-08-10 14:43:09 on dist-test-slave-h6n0"
server_key: "f5d6da2611e5f8ba6837db1e5ec77553"
server_key_iv: "89b11a2dd8fd1aab30a995a8ca151812"
server_key_version: "encryptionkey@0"
I20260810 14:43:09.337615 21433 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:43:09.341176 21453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:09.342765 21433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:43:09.342959 21433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "5bce0251e3674dea81f6ffb6b96467aa"
format_stamp: "Formatted at 2026-08-10 14:43:09 on dist-test-slave-h6n0"
server_key: "f5d6da2611e5f8ba6837db1e5ec77553"
server_key_iv: "89b11a2dd8fd1aab30a995a8ca151812"
server_key_version: "encryptionkey@0"
I20260810 14:43:09.343194 21433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:09.361508 21433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:09.362401 21433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:09.362737 21433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:09.385712 21433 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:46213
I20260810 14:43:09.385720 21536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:46213 every 8 connection(s)
I20260810 14:43:09.387121 21433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:09.391125 21537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:09.392489 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 21433
I20260810 14:43:09.392688 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:09.393314 21725 external_mini_cluster.cc:1428] Setting key dffcf00c3bcfd290421df13474ed5f79
I20260810 14:43:09.403419 21537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa: Bootstrap starting.
I20260810 14:43:09.406953 21537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:09.408056 21537 log.cc:824] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:09.409466 21540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:09.395600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51368 (local address 127.21.55.126:46213)
0810 14:43:09.396851 (+ 1251us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:09.396865 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:09.396920 (+ 55us) server_negotiation.cc:407] Connection header received
0810 14:43:09.398963 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:09.398987 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:09.399441 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:09.399797 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:09.400391 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:09.401676 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:09.403628 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:09.404327 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:09.404472 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:09.405788 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:09.405796 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:09.405807 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:09.406427 (+ 620us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:09.406566 (+ 139us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:09.406599 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:09.406688 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:09.407282 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 14:43:09.407716 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":517,"threads_started":1}
I20260810 14:43:09.411768 21537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa: No bootstrap required, opened a new log
I20260810 14:43:09.419063 21537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bce0251e3674dea81f6ffb6b96467aa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46213 } }
I20260810 14:43:09.419374 21537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:09.419461 21537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bce0251e3674dea81f6ffb6b96467aa, State: Initialized, Role: FOLLOWER
I20260810 14:43:09.419909 21537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bce0251e3674dea81f6ffb6b96467aa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46213 } }
I20260810 14:43:09.420100 21537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:09.420185 21537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:09.420302 21537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:09.422309 21537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bce0251e3674dea81f6ffb6b96467aa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46213 } }
I20260810 14:43:09.422838 21537 leader_election.cc:302] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bce0251e3674dea81f6ffb6b96467aa; no voters:
I20260810 14:43:09.423478 21537 leader_election.cc:288] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:09.423604 21543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:09.424363 21543 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [term 1 LEADER]: Becoming Leader. State: Replica: 5bce0251e3674dea81f6ffb6b96467aa, State: Running, Role: LEADER
I20260810 14:43:09.425377 21543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bce0251e3674dea81f6ffb6b96467aa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46213 } }
I20260810 14:43:09.426211 21537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:09.428294 21545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bce0251e3674dea81f6ffb6b96467aa. Latest consensus state: current_term: 1 leader_uuid: "5bce0251e3674dea81f6ffb6b96467aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bce0251e3674dea81f6ffb6b96467aa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46213 } } }
I20260810 14:43:09.428737 21545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:09.428689 21544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bce0251e3674dea81f6ffb6b96467aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bce0251e3674dea81f6ffb6b96467aa" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 46213 } } }
I20260810 14:43:09.429112 21544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:09.429831 21551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:09.435400 21551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:09.445478 21551 catalog_manager.cc:1347] Generated new cluster ID: 6c2283068b634da7a35e9dee1adeb43f
I20260810 14:43:09.445605 21551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:09.466650 21551 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:09.468089 21551 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:09.476765 21551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa: Generated new TSK 0
I20260810 14:43:09.477665 21551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:09.483778 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.65:0
--local_ip_for_outbound_sockets=127.21.55.65
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46213
--builtin_ntp_servers=127.21.55.84:34537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:43:09.881539 21572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:09.881879 21572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:09.882062 21572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:09.889477 21572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:09.889623 21572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:09.889808 21572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.65
I20260810 14:43:09.900188 21572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.55.65
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46213
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.65
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:09.901443 21572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:09.903259 21572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:09.916658 21579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:09.916767 21582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:09.916853 21580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:09.917734 21572 server_base.cc:1034] running on GCE node
I20260810 14:43:09.918574 21572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:09.919821 21572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:09.921142 21572 hybrid_clock.cc:648] HybridClock initialized: now 1786372989921072 us; error 61 us; skew 500 ppm
I20260810 14:43:09.921681 21572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:43:09.924887 21572 webserver.cc:460] Webserver started at http://127.21.55.65:36117/ using document root <none> and password file <none>
I20260810 14:43:09.925725 21572 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:09.925872 21572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:09.926301 21572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:09.929919 21572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/data/instance:
uuid: "a2556f398d6e4b26abae94e247b86fe7"
format_stamp: "Formatted at 2026-08-10 14:43:09 on dist-test-slave-h6n0"
server_key: "5a1d4eda3954558f699e5865ec5a4e2a"
server_key_iv: "44aff6e772b7f93856bf4f8373508279"
server_key_version: "encryptionkey@0"
I20260810 14:43:09.930938 21572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance:
uuid: "a2556f398d6e4b26abae94e247b86fe7"
format_stamp: "Formatted at 2026-08-10 14:43:09 on dist-test-slave-h6n0"
server_key: "5a1d4eda3954558f699e5865ec5a4e2a"
server_key_iv: "44aff6e772b7f93856bf4f8373508279"
server_key_version: "encryptionkey@0"
I20260810 14:43:09.937094 21572 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260810 14:43:09.941603 21592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:09.942895 21572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 14:43:09.943091 21572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/wal
uuid: "a2556f398d6e4b26abae94e247b86fe7"
format_stamp: "Formatted at 2026-08-10 14:43:09 on dist-test-slave-h6n0"
server_key: "5a1d4eda3954558f699e5865ec5a4e2a"
server_key_iv: "44aff6e772b7f93856bf4f8373508279"
server_key_version: "encryptionkey@0"
I20260810 14:43:09.943315 21572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:09.961431 21572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:09.962433 21572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:09.963088 21572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:09.964699 21572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:43:09.966749 21572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:43:09.966897 21572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:09.967052 21572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:43:09.967154 21572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:10.010716 21572 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.65:42541
I20260810 14:43:10.010764 21780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.65:42541 every 8 connection(s)
I20260810 14:43:10.012039 21572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/data/info.pb
I20260810 14:43:10.015938 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 21572
I20260810 14:43:10.016142 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-0/wal/instance
I20260810 14:43:10.016765 21725 external_mini_cluster.cc:1428] Setting key 703764f0137e7fa543b4724fc6706400
I20260810 14:43:10.019876 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.66:0
--local_ip_for_outbound_sockets=127.21.55.66
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46213
--builtin_ntp_servers=127.21.55.84:34537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:43:10.026196 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:10.014724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.65:43595 (local address 127.21.55.126:46213)
0810 14:43:10.015230 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:10.015235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:10.016610 (+ 1375us) server_negotiation.cc:407] Connection header received
0810 14:43:10.018824 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:10.018830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:10.018955 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:10.019081 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:10.020167 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:10.020848 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:10.022396 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:10.022879 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:10.023021 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:10.025257 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:10.025270 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:10.025274 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:10.025293 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:10.025339 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:10.025352 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:10.025466 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:10.025723 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 14:43:10.025821 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":283,"threads_started":1}
I20260810 14:43:10.027310 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:10.015193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46213 (local address 127.21.55.65:43595)
0810 14:43:10.016370 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:10.016433 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:10.018474 (+ 2041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:10.019299 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:10.019307 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:10.019388 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:10.019940 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:10.019956 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:10.021015 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:10.021028 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:10.022237 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:10.022249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:10.023125 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:10.023132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:10.023355 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:10.024433 (+ 1078us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:43:10.024438 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:10.024500 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:43:10.024624 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:10.025473 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:10.025486 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:10.025496 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:43:10.025711 (+ 215us) client_negotiation.cc:240] Negotiation successful
0810 14:43:10.026122 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":338,"threads_started":1}
I20260810 14:43:10.029069 21781 heartbeater.cc:347] Connected to a master server at 127.21.55.126:46213
I20260810 14:43:10.029838 21781 heartbeater.cc:464] Registering TS with master...
I20260810 14:43:10.031093 21781 heartbeater.cc:511] Master 127.21.55.126:46213 requested a full tablet report, sending...
I20260810 14:43:10.034588 21486 ts_manager.cc:194] Registered new tserver with Master: a2556f398d6e4b26abae94e247b86fe7 (127.21.55.65:42541)
I20260810 14:43:10.036150 21486 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.65:43595
W20260810 14:43:10.409754 21788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:10.410346 21788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:10.410537 21788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:10.417778 21788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:10.417870 21788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:10.418048 21788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.66
I20260810 14:43:10.427865 21788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.55.66
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46213
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.66
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:10.429198 21788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:10.431012 21788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:10.444852 21799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:10.444903 21797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:10.445325 21795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:10.446346 21788 server_base.cc:1034] running on GCE node
I20260810 14:43:10.447566 21788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:10.448793 21788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:10.450080 21788 hybrid_clock.cc:648] HybridClock initialized: now 1786372990450043 us; error 45 us; skew 500 ppm
I20260810 14:43:10.450441 21788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:43:10.453220 21788 webserver.cc:460] Webserver started at http://127.21.55.66:35699/ using document root <none> and password file <none>
I20260810 14:43:10.454083 21788 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:10.454226 21788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:10.454618 21788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:10.457651 21788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/data/instance:
uuid: "d60ca65f85a743a18f54315cd71a5d64"
format_stamp: "Formatted at 2026-08-10 14:43:10 on dist-test-slave-h6n0"
server_key: "8e0e77c2c66399445b2a4a36505dd341"
server_key_iv: "9e5ece558a34d38117024b2f05e478ff"
server_key_version: "encryptionkey@0"
I20260810 14:43:10.458467 21788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance:
uuid: "d60ca65f85a743a18f54315cd71a5d64"
format_stamp: "Formatted at 2026-08-10 14:43:10 on dist-test-slave-h6n0"
server_key: "8e0e77c2c66399445b2a4a36505dd341"
server_key_iv: "9e5ece558a34d38117024b2f05e478ff"
server_key_version: "encryptionkey@0"
I20260810 14:43:10.464124 21788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 14:43:10.469228 21808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:10.471078 21788 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 14:43:10.471299 21788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/wal
uuid: "d60ca65f85a743a18f54315cd71a5d64"
format_stamp: "Formatted at 2026-08-10 14:43:10 on dist-test-slave-h6n0"
server_key: "8e0e77c2c66399445b2a4a36505dd341"
server_key_iv: "9e5ece558a34d38117024b2f05e478ff"
server_key_version: "encryptionkey@0"
I20260810 14:43:10.471555 21788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:10.494971 21788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:10.495844 21788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:10.496172 21788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:10.497220 21788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:43:10.498972 21788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:43:10.499419 21788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:10.499663 21788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:43:10.499778 21788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:10.541356 21788 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.66:40759
I20260810 14:43:10.541455 22000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.66:40759 every 8 connection(s)
I20260810 14:43:10.542768 21788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/data/info.pb
I20260810 14:43:10.544407 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 21788
I20260810 14:43:10.544607 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/wal/instance
I20260810 14:43:10.545329 21725 external_mini_cluster.cc:1428] Setting key a4245de8ec49b36e7100601c7a77f96b
I20260810 14:43:10.551443 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.55.67:0
--local_ip_for_outbound_sockets=127.21.55.67
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46213
--builtin_ntp_servers=127.21.55.84:34537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:43:10.558878 22006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:10.546134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:44971 (local address 127.21.55.126:46213)
0810 14:43:10.548923 (+ 2789us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:10.548931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:10.548956 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:43:10.551068 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:10.551076 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:10.551171 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:10.551314 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:10.552601 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:10.553809 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:10.555549 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:10.556254 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:10.556410 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:10.557862 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:10.557871 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:10.557878 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:10.557903 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:10.557957 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:10.557976 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:10.558075 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:10.558293 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 14:43:10.558400 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2638,"thread_start_us":2509,"threads_started":1}
I20260810 14:43:10.560873 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:10.546634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46213 (local address 127.21.55.66:44971)
0810 14:43:10.548383 (+ 1749us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:10.548447 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:10.550733 (+ 2286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:10.551636 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:10.551650 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:10.551771 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:10.552401 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:10.552421 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:10.554007 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:10.554011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:10.555401 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:10.555414 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:10.556502 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:10.556508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:10.556716 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:10.557518 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:43:10.557521 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:10.557562 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:43:10.557650 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:10.558083 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:10.558092 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:10.558098 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:43:10.558250 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 14:43:10.559060 (+ 810us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"thread_start_us":628,"threads_started":1}
I20260810 14:43:10.562791 22001 heartbeater.cc:347] Connected to a master server at 127.21.55.126:46213
I20260810 14:43:10.563160 22001 heartbeater.cc:464] Registering TS with master...
I20260810 14:43:10.563943 22001 heartbeater.cc:511] Master 127.21.55.126:46213 requested a full tablet report, sending...
I20260810 14:43:10.565528 21486 ts_manager.cc:194] Registered new tserver with Master: d60ca65f85a743a18f54315cd71a5d64 (127.21.55.66:40759)
I20260810 14:43:10.566370 21486 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.66:44971
W20260810 14:43:10.926774 22008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:10.927296 22008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:10.927493 22008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:10.937040 22008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:10.937264 22008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:10.937484 22008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.55.67
I20260810 14:43:10.949400 22008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:34537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.55.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.55.67
--webserver_port=0
--tserver_master_addrs=127.21.55.126:46213
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.55.67
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:10.951588 22008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:10.954727 22008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:10.970866 22018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:10.970819 22021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:10.970916 22017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:10.976629 22008 server_base.cc:1034] running on GCE node
I20260810 14:43:10.977313 22008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:10.978379 22008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:10.979598 22008 hybrid_clock.cc:648] HybridClock initialized: now 1786372990979559 us; error 39 us; skew 500 ppm
I20260810 14:43:10.979947 22008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:43:10.987993 22008 webserver.cc:460] Webserver started at http://127.21.55.67:32829/ using document root <none> and password file <none>
I20260810 14:43:10.988746 22008 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:10.988844 22008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:10.989173 22008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:10.992126 22008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/data/instance:
uuid: "9567f428f0de403ba5f316bd698c5383"
format_stamp: "Formatted at 2026-08-10 14:43:10 on dist-test-slave-h6n0"
server_key: "7c949702e8cb3fd8cf43ea5f6f3dcb8f"
server_key_iv: "e7d1010d7acc42fd16b473867e667163"
server_key_version: "encryptionkey@0"
I20260810 14:43:10.992935 22008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance:
uuid: "9567f428f0de403ba5f316bd698c5383"
format_stamp: "Formatted at 2026-08-10 14:43:10 on dist-test-slave-h6n0"
server_key: "7c949702e8cb3fd8cf43ea5f6f3dcb8f"
server_key_iv: "e7d1010d7acc42fd16b473867e667163"
server_key_version: "encryptionkey@0"
I20260810 14:43:10.998139 22008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:43:11.002260 22027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:11.003774 22008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:43:11.003945 22008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/wal
uuid: "9567f428f0de403ba5f316bd698c5383"
format_stamp: "Formatted at 2026-08-10 14:43:10 on dist-test-slave-h6n0"
server_key: "7c949702e8cb3fd8cf43ea5f6f3dcb8f"
server_key_iv: "e7d1010d7acc42fd16b473867e667163"
server_key_version: "encryptionkey@0"
I20260810 14:43:11.004185 22008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:11.040481 21781 heartbeater.cc:503] Master 127.21.55.126:46213 was elected leader, sending a full tablet report...
I20260810 14:43:11.045488 22008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:11.046938 22008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:11.047678 22008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:11.048897 22008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:43:11.050571 22008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:43:11.050702 22008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:11.050848 22008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:43:11.050940 22008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:11.091820 22008 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.67:41395
I20260810 14:43:11.091917 22220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.67:41395 every 8 connection(s)
I20260810 14:43:11.093217 22008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/data/info.pb
I20260810 14:43:11.096788 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 22008
I20260810 14:43:11.096982 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/wal/instance
I20260810 14:43:11.097662 21725 external_mini_cluster.cc:1428] Setting key 56bebd28c2e115f2e569c0754517e1a5
I20260810 14:43:11.109273 22224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.096988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.67:60131 (local address 127.21.55.126:46213)
0810 14:43:11.097546 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:11.097551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:11.099223 (+ 1672us) server_negotiation.cc:407] Connection header received
0810 14:43:11.102448 (+ 3225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:11.102455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:11.102569 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:11.102704 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:11.103881 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.104654 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.106248 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.106712 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.106844 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.108365 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:11.108370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:11.108373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:11.108392 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:11.108432 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:11.108444 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:11.108565 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:11.108771 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 14:43:11.108877 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":295,"threads_started":1}
I20260810 14:43:11.110249 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.097777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:46213 (local address 127.21.55.67:60131)
0810 14:43:11.099053 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.099104 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:11.101921 (+ 2817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:11.102979 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:11.102989 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:11.103096 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:11.103690 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.103707 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.104826 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.104831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.106099 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.106112 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.106948 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.106957 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.107200 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.108010 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:43:11.108013 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:11.108054 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:43:11.108157 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:11.108544 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:11.108556 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:11.108563 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 14:43:11.108759 (+ 196us) client_negotiation.cc:240] Negotiation successful
0810 14:43:11.109159 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":393,"threads_started":1}
I20260810 14:43:11.111681 22222 heartbeater.cc:347] Connected to a master server at 127.21.55.126:46213
I20260810 14:43:11.111967 22222 heartbeater.cc:464] Registering TS with master...
I20260810 14:43:11.112891 22222 heartbeater.cc:511] Master 127.21.55.126:46213 requested a full tablet report, sending...
I20260810 14:43:11.114727 21484 ts_manager.cc:194] Registered new tserver with Master: 9567f428f0de403ba5f316bd698c5383 (127.21.55.67:41395)
I20260810 14:43:11.115712 21484 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.55.67:60131
I20260810 14:43:11.127305 21725 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:43:11.138862 22224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.131781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.21.55.126:46213)
0810 14:43:11.132028 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:11.132038 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:11.132501 (+ 463us) server_negotiation.cc:407] Connection header received
0810 14:43:11.132657 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:11.132664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:11.132772 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:11.132920 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:11.133787 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.134635 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.136086 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.136512 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.136621 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.137754 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:11.137759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:11.137762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:11.137780 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:11.137811 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:11.137823 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:11.137901 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:11.138304 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 14:43:11.138484 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:43:11.144357 21484 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:43:11.151888 21484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:43:11.180934 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.167950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41756 (local address 127.21.55.67:41395)
0810 14:43:11.168862 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:11.168871 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:11.169880 (+ 1009us) server_negotiation.cc:407] Connection header received
0810 14:43:11.170323 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:11.170331 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:11.170637 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:11.170821 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:11.171915 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.173564 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.177111 (+ 3547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.179160 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.179280 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.179579 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:11.180313 (+ 734us) server_negotiation.cc:299] Negotiation successful
0810 14:43:11.180518 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":451,"threads_started":1}
I20260810 14:43:11.182060 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.167704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:41395 (local address 127.0.0.1:41756)
0810 14:43:11.169711 (+ 2007us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.169761 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:11.170056 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:11.171171 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:11.171178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:11.171209 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:11.171690 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.171706 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.173759 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.173764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.176803 (+ 3039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.176824 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.179801 (+ 2977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.179809 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.180029 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.180093 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 14:43:11.181493 (+ 1400us) client_negotiation.cc:240] Negotiation successful
0810 14:43:11.181731 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1736,"thread_start_us":2310,"threads_started":1}
I20260810 14:43:11.183753 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.167965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42541 (local address 127.0.0.1:33748)
0810 14:43:11.170135 (+ 2170us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.170155 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:11.170260 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:11.171406 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:11.171412 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:11.171439 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:11.173084 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.173100 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.177182 (+ 4082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.177190 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.179683 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.179700 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.183006 (+ 3306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.183013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.183214 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.183251 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 14:43:11.183316 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 14:43:11.183442 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2030,"thread_start_us":1188,"threads_started":1}
I20260810 14:43:11.185123 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.168450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.21.55.65:42541)
0810 14:43:11.170828 (+ 2378us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:11.170836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:11.170862 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:43:11.170960 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:11.170967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:11.171186 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:11.173285 (+ 2099us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:11.173525 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.176983 (+ 3458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.179979 (+ 2996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.182771 (+ 2792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.184227 (+ 1456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.184464 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:11.184518 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 14:43:11.184724 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2182,"thread_start_us":2011,"threads_started":1}
I20260810 14:43:11.188370 22115 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef11d2552e345e581d45ae4877abc87 (DEFAULT_TABLE table=test-table [id=a6a471fa051d4a10bdc87c0447e13e5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:43:11.188242 21670 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef11d2552e345e581d45ae4877abc87 (DEFAULT_TABLE table=test-table [id=a6a471fa051d4a10bdc87c0447e13e5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:43:11.189648 22115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef11d2552e345e581d45ae4877abc87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:11.189709 21670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef11d2552e345e581d45ae4877abc87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:11.190703 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.170499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.66:40759 (local address 127.0.0.1:36626)
0810 14:43:11.182463 (+ 11964us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.182483 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:11.182613 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:11.183793 (+ 1180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:11.183801 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:11.183838 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:11.184073 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.184088 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.186305 (+ 2217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.186311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.188204 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.188221 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.189943 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.189950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.190159 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.190193 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 14:43:11.190296 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 14:43:11.190425 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11843,"thread_start_us":13568,"threads_started":1}
I20260810 14:43:11.190933 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.170968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36626 (local address 127.21.55.66:40759)
0810 14:43:11.183158 (+ 12190us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:11.183166 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:11.183195 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:43:11.183306 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:11.183315 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:11.183541 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:11.183763 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:11.184302 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.186126 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.188376 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.189784 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.189921 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.190165 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:11.190352 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 14:43:11.190514 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12016,"thread_start_us":11873,"threads_started":1}
I20260810 14:43:11.195353 21894 tablet_service.cc:1467] Processing CreateTablet for tablet 7ef11d2552e345e581d45ae4877abc87 (DEFAULT_TABLE table=test-table [id=a6a471fa051d4a10bdc87c0447e13e5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:43:11.196800 21894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ef11d2552e345e581d45ae4877abc87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:11.201563 22256 tablet_bootstrap.cc:492] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7: Bootstrap starting.
I20260810 14:43:11.204830 22256 tablet_bootstrap.cc:654] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:11.205153 22257 tablet_bootstrap.cc:492] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383: Bootstrap starting.
I20260810 14:43:11.206296 22256 log.cc:824] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:11.208851 22257 tablet_bootstrap.cc:654] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:11.210047 22257 log.cc:824] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:11.210582 22256 tablet_bootstrap.cc:492] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7: No bootstrap required, opened a new log
I20260810 14:43:11.211017 22256 ts_tablet_manager.cc:1397] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 14:43:11.212821 22265 tablet_bootstrap.cc:492] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: Bootstrap starting.
I20260810 14:43:11.213254 22257 tablet_bootstrap.cc:492] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383: No bootstrap required, opened a new log
I20260810 14:43:11.213574 22257 ts_tablet_manager.cc:1397] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260810 14:43:11.216073 22265 tablet_bootstrap.cc:654] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:11.217001 22265 log.cc:824] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:11.219801 22265 tablet_bootstrap.cc:492] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: No bootstrap required, opened a new log
I20260810 14:43:11.220069 22265 ts_tablet_manager.cc:1397] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 14:43:11.222141 22256 raft_consensus.cc:348] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f428f0de403ba5f316bd698c5383" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41395 } } peers { permanent_uuid: "d60ca65f85a743a18f54315cd71a5d64" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40759 } } peers { permanent_uuid: "a2556f398d6e4b26abae94e247b86fe7" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42541 } }
I20260810 14:43:11.222591 22256 raft_consensus.cc:374] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:11.222690 22256 raft_consensus.cc:729] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2556f398d6e4b26abae94e247b86fe7, State: Initialized, Role: FOLLOWER
I20260810 14:43:11.223315 22256 consensus_queue.cc:260] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f428f0de403ba5f316bd698c5383" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41395 } } peers { permanent_uuid: "d60ca65f85a743a18f54315cd71a5d64" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40759 } } peers { permanent_uuid: "a2556f398d6e4b26abae94e247b86fe7" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42541 } }
I20260810 14:43:11.223876 22257 raft_consensus.cc:348] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f428f0de403ba5f316bd698c5383" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41395 } } peers { permanent_uuid: "d60ca65f85a743a18f54315cd71a5d64" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40759 } } peers { permanent_uuid: "a2556f398d6e4b26abae94e247b86fe7" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42541 } }
I20260810 14:43:11.224292 22257 raft_consensus.cc:374] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:11.224385 22257 raft_consensus.cc:729] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9567f428f0de403ba5f316bd698c5383, State: Initialized, Role: FOLLOWER
I20260810 14:43:11.225165 22256 ts_tablet_manager.cc:1428] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260810 14:43:11.225070 22257 consensus_queue.cc:260] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f428f0de403ba5f316bd698c5383" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41395 } } peers { permanent_uuid: "d60ca65f85a743a18f54315cd71a5d64" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40759 } } peers { permanent_uuid: "a2556f398d6e4b26abae94e247b86fe7" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42541 } }
I20260810 14:43:11.226207 22222 heartbeater.cc:503] Master 127.21.55.126:46213 was elected leader, sending a full tablet report...
I20260810 14:43:11.226784 22257 ts_tablet_manager.cc:1428] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260810 14:43:11.229079 22265 raft_consensus.cc:348] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f428f0de403ba5f316bd698c5383" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41395 } } peers { permanent_uuid: "d60ca65f85a743a18f54315cd71a5d64" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40759 } } peers { permanent_uuid: "a2556f398d6e4b26abae94e247b86fe7" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42541 } }
I20260810 14:43:11.229996 22265 raft_consensus.cc:374] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:11.230147 22265 raft_consensus.cc:729] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60ca65f85a743a18f54315cd71a5d64, State: Initialized, Role: FOLLOWER
I20260810 14:43:11.230773 22265 consensus_queue.cc:260] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f428f0de403ba5f316bd698c5383" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41395 } } peers { permanent_uuid: "d60ca65f85a743a18f54315cd71a5d64" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40759 } } peers { permanent_uuid: "a2556f398d6e4b26abae94e247b86fe7" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42541 } }
I20260810 14:43:11.231835 22001 heartbeater.cc:503] Master 127.21.55.126:46213 was elected leader, sending a full tablet report...
I20260810 14:43:11.232647 22265 ts_tablet_manager.cc:1428] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260810 14:43:11.232862 22273 raft_consensus.cc:482] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:43:11.233129 22273 raft_consensus.cc:504] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f428f0de403ba5f316bd698c5383" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41395 } } peers { permanent_uuid: "d60ca65f85a743a18f54315cd71a5d64" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40759 } } peers { permanent_uuid: "a2556f398d6e4b26abae94e247b86fe7" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42541 } }
I20260810 14:43:11.234908 22273 leader_election.cc:288] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9567f428f0de403ba5f316bd698c5383 (127.21.55.67:41395), a2556f398d6e4b26abae94e247b86fe7 (127.21.55.65:42541)
I20260810 14:43:11.243062 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.236075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:54643 (local address 127.21.55.67:41395)
0810 14:43:11.237337 (+ 1262us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:11.237348 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:11.237376 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:43:11.237465 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:11.237469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:11.237591 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:11.237757 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:11.238516 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.239473 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.241349 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.242111 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.242290 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.242363 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:11.242680 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 14:43:11.242803 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1046}
I20260810 14:43:11.243104 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.235270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.67:41395 (local address 127.21.55.66:54643)
0810 14:43:11.236372 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.236391 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:11.236531 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:11.237816 (+ 1285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:11.237823 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:11.237856 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:11.238306 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.238323 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.239626 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.239634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.241138 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.241151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.242317 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.242323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.242480 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.242504 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 14:43:11.242654 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 14:43:11.242782 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"thread_start_us":1013,"threads_started":1}
I20260810 14:43:11.243865 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.235899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.55.66:43871 (local address 127.21.55.65:42541)
0810 14:43:11.236269 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:11.236275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:11.237133 (+ 858us) server_negotiation.cc:407] Connection header received
0810 14:43:11.237311 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:11.237316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:11.237405 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:11.238782 (+ 1377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:43:11.239571 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.240467 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.242120 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.243022 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.243189 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.243252 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:11.243522 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 14:43:11.243641 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229}
I20260810 14:43:11.243888 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.235894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42541 (local address 127.21.55.66:43871)
0810 14:43:11.237029 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.237043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:11.237181 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:11.239011 (+ 1830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:11.239016 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:11.239039 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:43:11.239404 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.239414 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.240603 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.240608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.241937 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:11.241947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.243213 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.243219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:11.243369 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.243391 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 14:43:11.243495 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 14:43:11.243608 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"thread_start_us":892,"threads_started":1}
I20260810 14:43:11.244374 22145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef11d2552e345e581d45ae4877abc87" candidate_uuid: "d60ca65f85a743a18f54315cd71a5d64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9567f428f0de403ba5f316bd698c5383" is_pre_election: true
I20260810 14:43:11.244889 22145 raft_consensus.cc:2454] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ca65f85a743a18f54315cd71a5d64 in term 0.
I20260810 14:43:11.244803 21698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef11d2552e345e581d45ae4877abc87" candidate_uuid: "d60ca65f85a743a18f54315cd71a5d64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2556f398d6e4b26abae94e247b86fe7" is_pre_election: true
I20260810 14:43:11.245254 21698 raft_consensus.cc:2454] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60ca65f85a743a18f54315cd71a5d64 in term 0.
I20260810 14:43:11.245556 21809 leader_election.cc:302] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9567f428f0de403ba5f316bd698c5383, d60ca65f85a743a18f54315cd71a5d64; no voters:
I20260810 14:43:11.245985 22273 raft_consensus.cc:2781] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:43:11.246563 22273 raft_consensus.cc:482] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:43:11.246757 22273 raft_consensus.cc:3037] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:11.250319 22273 raft_consensus.cc:504] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f428f0de403ba5f316bd698c5383" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41395 } } peers { permanent_uuid: "d60ca65f85a743a18f54315cd71a5d64" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40759 } } peers { permanent_uuid: "a2556f398d6e4b26abae94e247b86fe7" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42541 } }
I20260810 14:43:11.251217 22273 leader_election.cc:288] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [CANDIDATE]: Term 1 election: Requested vote from peers 9567f428f0de403ba5f316bd698c5383 (127.21.55.67:41395), a2556f398d6e4b26abae94e247b86fe7 (127.21.55.65:42541)
I20260810 14:43:11.251646 22145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef11d2552e345e581d45ae4877abc87" candidate_uuid: "d60ca65f85a743a18f54315cd71a5d64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9567f428f0de403ba5f316bd698c5383"
I20260810 14:43:11.251771 21698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ef11d2552e345e581d45ae4877abc87" candidate_uuid: "d60ca65f85a743a18f54315cd71a5d64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2556f398d6e4b26abae94e247b86fe7"
I20260810 14:43:11.251874 22145 raft_consensus.cc:3037] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:11.251983 21698 raft_consensus.cc:3037] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:11.255216 22145 raft_consensus.cc:2454] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ca65f85a743a18f54315cd71a5d64 in term 1.
I20260810 14:43:11.255216 21698 raft_consensus.cc:2454] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60ca65f85a743a18f54315cd71a5d64 in term 1.
I20260810 14:43:11.255682 21811 leader_election.cc:302] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2556f398d6e4b26abae94e247b86fe7, d60ca65f85a743a18f54315cd71a5d64; no voters:
I20260810 14:43:11.256141 22273 raft_consensus.cc:2781] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:11.256402 22273 raft_consensus.cc:686] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 1 LEADER]: Becoming Leader. State: Replica: d60ca65f85a743a18f54315cd71a5d64, State: Running, Role: LEADER
I20260810 14:43:11.257113 22273 consensus_queue.cc:237] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f428f0de403ba5f316bd698c5383" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41395 } } peers { permanent_uuid: "d60ca65f85a743a18f54315cd71a5d64" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40759 } } peers { permanent_uuid: "a2556f398d6e4b26abae94e247b86fe7" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42541 } }
I20260810 14:43:11.262490 21486 catalog_manager.cc:5495] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 reported cstate change: term changed from 0 to 1, leader changed from <none> to d60ca65f85a743a18f54315cd71a5d64 (127.21.55.66). New cstate: current_term: 1 leader_uuid: "d60ca65f85a743a18f54315cd71a5d64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9567f428f0de403ba5f316bd698c5383" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60ca65f85a743a18f54315cd71a5d64" member_type: VOTER last_known_addr { host: "127.21.55.66" port: 40759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2556f398d6e4b26abae94e247b86fe7" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42541 } health_report { overall_health: UNKNOWN } } }
W20260810 14:43:11.264662 21782 tablet.cc:2369] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:43:11.283121 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.277070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36642 (local address 127.21.55.66:40759)
0810 14:43:11.277289 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:11.277298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:11.277355 (+ 57us) server_negotiation.cc:407] Connection header received
0810 14:43:11.277567 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:11.277570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:11.277652 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:11.277969 (+ 317us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:43:11.278192 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.278962 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.280432 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.280947 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.281126 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.281658 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:43:11.282346 (+ 688us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:43:11.282490 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:11.282666 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 14:43:11.282752 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:43:11.292538 21698 raft_consensus.cc:1261] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7 [term 1 FOLLOWER]: Refusing update from remote peer d60ca65f85a743a18f54315cd71a5d64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:43:11.293346 22145 raft_consensus.cc:1261] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383 [term 1 FOLLOWER]: Refusing update from remote peer d60ca65f85a743a18f54315cd71a5d64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:43:11.293570 22271 consensus_queue.cc:1035] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2556f398d6e4b26abae94e247b86fe7" member_type: VOTER last_known_addr { host: "127.21.55.65" port: 42541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:43:11.294512 22271 consensus_queue.cc:1035] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9567f428f0de403ba5f316bd698c5383" member_type: VOTER last_known_addr { host: "127.21.55.67" port: 41395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 14:43:11.294816 22003 tablet.cc:2369] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:43:11.307067 22284 mvcc.cc:205] Tried to move back new op lower bound from 7316983772324012032 to 7316983772193890304. Current Snapshot: MvccSnapshot[applied={T|T < 7316983772324012032}]
I20260810 14:43:11.323553 21486 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:43:11.323817 21486 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:43:11.327907 21486 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5bce0251e3674dea81f6ffb6b96467aa: Sending DeleteTablet for 3 replicas of tablet 7ef11d2552e345e581d45ae4877abc87
I20260810 14:43:11.329658 22115 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef11d2552e345e581d45ae4877abc87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:43:11 UTC) from {username='slave'} at 127.0.0.1:41756
I20260810 14:43:11.329663 21670 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef11d2552e345e581d45ae4877abc87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:43:11 UTC) from {username='slave'} at 127.0.0.1:33748
I20260810 14:43:11.330299 21894 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ef11d2552e345e581d45ae4877abc87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:43:11 UTC) from {username='slave'} at 127.0.0.1:36626
I20260810 14:43:11.332230 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 21572
I20260810 14:43:11.347609 22300 tablet_replica.cc:330] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7: stopping tablet replica
I20260810 14:43:11.348511 22302 tablet_replica.cc:330] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: stopping tablet replica
I20260810 14:43:11.348340 22300 raft_consensus.cc:2229] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:43:11.349327 22302 raft_consensus.cc:2229] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:43:11.349629 22300 raft_consensus.cc:2258] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:43:11.350123 22302 raft_consensus.cc:2258] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:43:11.351989 22300 ts_tablet_manager.cc:1905] T 7ef11d2552e345e581d45ae4877abc87 P a2556f398d6e4b26abae94e247b86fe7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:43:11.355607 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.333244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.21.55.65:42541)
0810 14:43:11.333460 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:11.333469 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:11.339590 (+ 6121us) server_negotiation.cc:407] Connection header received
0810 14:43:11.340529 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:11.340538 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:11.340707 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:11.340802 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:11.342172 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.343797 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.350834 (+ 7037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.351497 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.351602 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.353206 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:11.353217 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:11.353239 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:11.354222 (+ 983us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:11.354337 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:11.354384 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:11.354592 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:11.355001 (+ 409us) server_negotiation.cc:299] Negotiation successful
0810 14:43:11.355162 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:43:11.357141 21775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:11.363277 22299 tablet_replica.cc:330] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383: stopping tablet replica
I20260810 14:43:11.364216 22299 raft_consensus.cc:2229] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:43:11.364818 22302 ts_tablet_manager.cc:1905] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:43:11.364847 22299 raft_consensus.cc:2258] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:43:11.366613 22299 ts_tablet_manager.cc:1905] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:43:11.373581 22302 ts_tablet_manager.cc:1918] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:43:11.373755 22302 log.cc:1196] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-1/wal/wals/7ef11d2552e345e581d45ae4877abc87
I20260810 14:43:11.374469 22302 ts_tablet_manager.cc:1939] T 7ef11d2552e345e581d45ae4877abc87 P d60ca65f85a743a18f54315cd71a5d64: Deleting consensus metadata
I20260810 14:43:11.374791 22299 ts_tablet_manager.cc:1918] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:43:11.374953 22299 log.cc:1196] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383: Deleting WAL directory at /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786372641103011-21725-0/minicluster-data/ts-2/wal/wals/7ef11d2552e345e581d45ae4877abc87
I20260810 14:43:11.375552 22299 ts_tablet_manager.cc:1939] T 7ef11d2552e345e581d45ae4877abc87 P 9567f428f0de403ba5f316bd698c5383: Deleting consensus metadata
I20260810 14:43:11.376655 21455 catalog_manager.cc:4841] TS d60ca65f85a743a18f54315cd71a5d64 (127.21.55.66:40759): tablet 7ef11d2552e345e581d45ae4877abc87 (table test-table [id=a6a471fa051d4a10bdc87c0447e13e5b]) successfully deleted
I20260810 14:43:11.377774 21455 catalog_manager.cc:4841] TS 9567f428f0de403ba5f316bd698c5383 (127.21.55.67:41395): tablet 7ef11d2552e345e581d45ae4877abc87 (table test-table [id=a6a471fa051d4a10bdc87c0447e13e5b]) successfully deleted
I20260810 14:43:11.493101 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 21572
I20260810 14:43:11.519313 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 21788
W20260810 14:43:11.519320 21462 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.55.65:42541 (error 108)
I20260810 14:43:11.521874 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.521394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42541 (local address 127.0.0.1:33768)
0810 14:43:11.521630 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.521723 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 14:43:11.522192 21462 catalog_manager.cc:4584] TS a2556f398d6e4b26abae94e247b86fe7 (127.21.55.65:42541): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ef11d2552e345e581d45ae4877abc87: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42541: connect: Connection refused (error 111)
I20260810 14:43:11.527976 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.520622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36658 (local address 127.21.55.66:40759)
0810 14:43:11.520816 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:11.520823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:11.520850 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:43:11.520930 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:11.520934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:11.521011 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:11.521108 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:11.521625 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.522871 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.524150 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.524776 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.524900 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.526060 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:11.526071 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:11.526089 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:11.527066 (+ 977us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:11.527128 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:11.527154 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:11.527274 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:11.527468 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 14:43:11.527577 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:43:11.528795 21994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:11.576195 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.575601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42541 (local address 127.0.0.1:33778)
0810 14:43:11.575940 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.576041 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 14:43:11.577260 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.576909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42541 (local address 127.0.0.1:33792)
0810 14:43:11.577072 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.577137 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 14:43:11.638418 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.637790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42541 (local address 127.0.0.1:33794)
0810 14:43:11.638119 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.638256 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260810 14:43:11.639756 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.639386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42541 (local address 127.0.0.1:33810)
0810 14:43:11.639545 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.639628 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 14:43:11.657341 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 21788
I20260810 14:43:11.693104 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22008
I20260810 14:43:11.701683 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.693957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.21.55.67:41395)
0810 14:43:11.694178 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:11.694182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:11.694206 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:43:11.694283 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:11.694287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:11.694396 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:11.694510 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:11.695250 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.696006 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.697512 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:11.698123 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:11.698306 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:11.700010 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:11.700021 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:11.700041 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:11.700855 (+ 814us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:11.700919 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:11.700954 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:11.701073 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:11.701268 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 14:43:11.701374 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 14:43:11.702477 22214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:11.720453 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.719954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42541 (local address 127.0.0.1:33822)
0810 14:43:11.720213 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.720306 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260810 14:43:11.721709 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.721288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42541 (local address 127.0.0.1:33828)
0810 14:43:11.721512 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.721594 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260810 14:43:11.794308 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 22008
I20260810 14:43:11.819977 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 21433
I20260810 14:43:11.821507 21531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:11.943176 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:11.942492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.65:42541 (local address 127.0.0.1:33838)
0810 14:43:11.942841 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:11.942949 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.65:42541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211}
I20260810 14:43:11.943602 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 21433
2026-08-10T14:43:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3135 ms)
[----------] 6 tests from EncryptionPolicyTest (24909 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:12 dist-test-slave-h6n0 krb5kdc[22318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:12 dist-test-slave-h6n0 krb5kdc[22318](info): set up 2 sockets
Aug 10 14:43:12 dist-test-slave-h6n0 krb5kdc[22318](info): commencing operation
krb5kdc: starting...
W20260810 14:43:13.364662 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.361s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:13 dist-test-slave-h6n0 krb5kdc[22318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:13.703042 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37667
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:14.060796 22335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:14.061121 22335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:14.061221 22335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:14.068136 22335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:14.068246 22335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:14.068292 22335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:14.068354 22335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:14.068398 22335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:14.079854 22335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37667
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:14.081205 22335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:14.082505 22335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:14.095619 22346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:14.095723 22340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:14.095741 22349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:14.096849 22335 server_base.cc:1034] running on GCE node
I20260810 14:43:14.097725 22335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:14.099184 22335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:14.100428 22335 hybrid_clock.cc:648] HybridClock initialized: now 1786372994100441 us; error 100 us; skew 500 ppm
I20260810 14:43:14.100797 22335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:14 dist-test-slave-h6n0 krb5kdc[22318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372994, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:14.104348 22335 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:14.106190 22335 webserver.cc:460] Webserver started at http://127.0.0.1:42945/ using document root <none> and password file <none>
I20260810 14:43:14.106927 22335 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:14.107229 22335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:14.107707 22335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:14.111758 22335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "38b5ef6afb0e467db9e7183b9ad4e185"
format_stamp: "Formatted at 2026-08-10 14:43:14 on dist-test-slave-h6n0"
server_key: "9f1f9ed248cca781d7a33e45cdc34c0b"
server_key_iv: "7ce808ef8ffdc560c32a9df7c6a115e4"
server_key_version: "encryptionkey@0"
I20260810 14:43:14.112641 22335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "38b5ef6afb0e467db9e7183b9ad4e185"
format_stamp: "Formatted at 2026-08-10 14:43:14 on dist-test-slave-h6n0"
server_key: "9f1f9ed248cca781d7a33e45cdc34c0b"
server_key_iv: "7ce808ef8ffdc560c32a9df7c6a115e4"
server_key_version: "encryptionkey@0"
I20260810 14:43:14.118971 22335 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 14:43:14.122834 22361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:14.124691 22335 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 14:43:14.124979 22335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "38b5ef6afb0e467db9e7183b9ad4e185"
format_stamp: "Formatted at 2026-08-10 14:43:14 on dist-test-slave-h6n0"
server_key: "9f1f9ed248cca781d7a33e45cdc34c0b"
server_key_iv: "7ce808ef8ffdc560c32a9df7c6a115e4"
server_key_version: "encryptionkey@0"
I20260810 14:43:14.125285 22335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:14.155435 22335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:14.161244 22335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:14.161731 22335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:14.183450 22335 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37667
I20260810 14:43:14.183571 22445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37667 every 8 connection(s)
I20260810 14:43:14.184846 22335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:14.188628 22448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:14.193608 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 22335
I20260810 14:43:14.193856 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:14.194593 21725 external_mini_cluster.cc:1428] Setting key b535b4f862e68dabfd89146fe7e96621
I20260810 14:43:14.199326 22448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185: Bootstrap starting.
I20260810 14:43:14.203015 22448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:14.204141 22448 log.cc:824] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:14.207211 22448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185: No bootstrap required, opened a new log
Aug 10 14:43:14 dist-test-slave-h6n0 krb5kdc[22318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 14:43:14.216562 22448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b5ef6afb0e467db9e7183b9ad4e185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } }
I20260810 14:43:14.217108 22448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:14.217270 22448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38b5ef6afb0e467db9e7183b9ad4e185, State: Initialized, Role: FOLLOWER
I20260810 14:43:14.218070 22448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b5ef6afb0e467db9e7183b9ad4e185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } }
I20260810 14:43:14.218380 22448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:14.218614 22448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:14.218854 22448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:14.220062 22452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:14.196627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53535 (local address 127.0.0.1:37667)
0810 14:43:14.197507 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:14.197521 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:14.197566 (+ 45us) server_negotiation.cc:407] Connection header received
0810 14:43:14.199540 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:14.199562 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:14.199920 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:14.200217 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:14.202166 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:14.203664 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:14.205368 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:14.206034 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:14.206167 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:14.211254 (+ 5087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:14.211263 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:14.211276 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:14.211330 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:14.214988 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:14.215885 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:14.215893 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:14.215898 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:14.216073 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:14.216623 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:14.216627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:14.216629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:14.217583 (+ 954us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:14.218186 (+ 603us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:14.218721 (+ 535us) server_negotiation.cc:299] Negotiation successful
0810 14:43:14.219082 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":435,"threads_started":1}
I20260810 14:43:14.221745 22448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b5ef6afb0e467db9e7183b9ad4e185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } }
I20260810 14:43:14.222318 22448 leader_election.cc:302] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38b5ef6afb0e467db9e7183b9ad4e185; no voters:
I20260810 14:43:14.223057 22448 leader_election.cc:288] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:14.223156 22454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:14.224262 22454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [term 1 LEADER]: Becoming Leader. State: Replica: 38b5ef6afb0e467db9e7183b9ad4e185, State: Running, Role: LEADER
I20260810 14:43:14.224387 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:14.224924 22454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b5ef6afb0e467db9e7183b9ad4e185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } }
I20260810 14:43:14.225394 22448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:14.227880 22455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38b5ef6afb0e467db9e7183b9ad4e185" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b5ef6afb0e467db9e7183b9ad4e185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } } }
I20260810 14:43:14.228349 22455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:14.228984 22456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38b5ef6afb0e467db9e7183b9ad4e185. Latest consensus state: current_term: 1 leader_uuid: "38b5ef6afb0e467db9e7183b9ad4e185" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38b5ef6afb0e467db9e7183b9ad4e185" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } } }
I20260810 14:43:14.229324 22456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:14.236840 22461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:14.243664 22461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:14.257345 22461 catalog_manager.cc:1347] Generated new cluster ID: d743c03d89ed410f88e74ba6fc67e0be
I20260810 14:43:14.257571 22461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:14.288010 22452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:14.247677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55803 (local address 127.0.0.1:37667)
0810 14:43:14.247840 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:14.247847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:14.253523 (+ 5676us) server_negotiation.cc:407] Connection header received
0810 14:43:14.253821 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:14.253830 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:14.253963 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:14.254107 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:14.256745 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:14.257983 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:14.259864 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:14.260523 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:14.260622 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:14.264173 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:14.264181 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:14.264186 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:14.264223 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:14.268113 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:14.270228 (+ 2115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:14.270234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:14.270237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:14.270396 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:14.285990 (+ 15594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:14.285997 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:14.286000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:14.286397 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:14.286730 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:14.287472 (+ 742us) server_negotiation.cc:299] Negotiation successful
0810 14:43:14.287670 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 14:43:14.289039 22461 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:14.290553 22461 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:14.304587 22461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38b5ef6afb0e467db9e7183b9ad4e185: Generated new TSK 0
I20260810 14:43:14.305389 22461 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:14.320276 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22335
I20260810 14:43:14.321517 22438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:14.414259 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 22335
W20260810 14:43:14.436806 22315 connection.cc:535] client connection to 127.0.0.1:37667 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37667): Connection reset by peer (error 104)
2026-08-10T14:43:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:14 dist-test-slave-h6n0 krb5kdc[22487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:14 dist-test-slave-h6n0 krb5kdc[22487](info): set up 2 sockets
Aug 10 14:43:14 dist-test-slave-h6n0 krb5kdc[22487](info): commencing operation
krb5kdc: starting...
W20260810 14:43:15.790473 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:15 dist-test-slave-h6n0 krb5kdc[22487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:16.111892 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:16.502162 22503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:16.502501 22503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:16.502718 22503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:16.509895 22503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:16.510051 22503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:16.510164 22503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:16.510279 22503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:16.510370 22503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:16.520248 22503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:16.521510 22503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:16.523029 22503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:16.536320 22511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:16.536381 22514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:16.536434 22510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:16.540244 22503 server_base.cc:1034] running on GCE node
I20260810 14:43:16.541039 22503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:16.542407 22503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:16.543639 22503 hybrid_clock.cc:648] HybridClock initialized: now 1786372996543603 us; error 28 us; skew 500 ppm
I20260810 14:43:16.544071 22503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:16 dist-test-slave-h6n0 krb5kdc[22487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372996, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:16.547526 22503 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:16.549386 22503 webserver.cc:460] Webserver started at http://127.0.0.1:38579/ using document root <none> and password file <none>
I20260810 14:43:16.550104 22503 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:16.550275 22503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:16.550693 22503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:16.553824 22503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "a1017ee465104cdd95975ba5a6a7c41d"
format_stamp: "Formatted at 2026-08-10 14:43:16 on dist-test-slave-h6n0"
server_key: "dabc0dd4b36f715e2965dd086ff72ed4"
server_key_iv: "15f5e7fbc90564ec24581244f2fd6ae2"
server_key_version: "encryptionkey@0"
I20260810 14:43:16.554709 22503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "a1017ee465104cdd95975ba5a6a7c41d"
format_stamp: "Formatted at 2026-08-10 14:43:16 on dist-test-slave-h6n0"
server_key: "dabc0dd4b36f715e2965dd086ff72ed4"
server_key_iv: "15f5e7fbc90564ec24581244f2fd6ae2"
server_key_version: "encryptionkey@0"
I20260810 14:43:16.559926 22503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:43:16.563508 22524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:16.564848 22503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:43:16.565045 22503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "a1017ee465104cdd95975ba5a6a7c41d"
format_stamp: "Formatted at 2026-08-10 14:43:16 on dist-test-slave-h6n0"
server_key: "dabc0dd4b36f715e2965dd086ff72ed4"
server_key_iv: "15f5e7fbc90564ec24581244f2fd6ae2"
server_key_version: "encryptionkey@0"
I20260810 14:43:16.565246 22503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:16.635428 22503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:16.641336 22503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:16.641695 22503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:16.663583 22503 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37225
I20260810 14:43:16.663733 22596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37225 every 8 connection(s)
I20260810 14:43:16.665095 22503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:16.669389 22598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:16.673303 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 22503
I20260810 14:43:16.673519 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:16.674094 21725 external_mini_cluster.cc:1428] Setting key f09627fe99455b74034ff72245dd04fe
I20260810 14:43:16.683077 22598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d: Bootstrap starting.
I20260810 14:43:16.686868 22598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:16.688324 22598 log.cc:824] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:16.689829 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:16.676229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33117 (local address 127.0.0.1:37225)
0810 14:43:16.677048 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:16.677060 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:16.677101 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:43:16.678881 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:16.678902 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:16.679309 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:16.679631 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:16.680254 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:16.681563 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:16.683539 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:16.684578 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:16.684948 (+ 370us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:16.686619 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:16.686629 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:16.686643 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:16.687262 (+ 619us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:16.687365 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:16.687395 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:16.687555 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:16.688086 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 14:43:16.688448 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":409,"threads_started":1}
I20260810 14:43:16.691373 22598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d: No bootstrap required, opened a new log
I20260810 14:43:16.693722 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:16.701190 22598 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1017ee465104cdd95975ba5a6a7c41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260810 14:43:16.701678 22598 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:16.701798 22598 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1017ee465104cdd95975ba5a6a7c41d, State: Initialized, Role: FOLLOWER
I20260810 14:43:16.702390 22598 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1017ee465104cdd95975ba5a6a7c41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260810 14:43:16.702731 22598 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:16.702891 22598 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:16.703066 22598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:16.705724 22598 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1017ee465104cdd95975ba5a6a7c41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260810 14:43:16.706336 22598 leader_election.cc:302] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1017ee465104cdd95975ba5a6a7c41d; no voters:
I20260810 14:43:16.706802 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:16.699385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47879 (local address 127.0.0.1:37225)
0810 14:43:16.699622 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:16.699629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:16.700464 (+ 835us) server_negotiation.cc:407] Connection header received
0810 14:43:16.700607 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:16.700612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:16.700688 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:16.700798 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:16.701357 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:16.702500 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:16.704003 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:16.704725 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:16.704864 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:16.705954 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:16.705960 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:16.705965 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:16.705983 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:16.706060 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:16.706083 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:16.706228 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:16.706412 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 14:43:16.706463 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 14:43:16.707137 22598 leader_election.cc:288] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:16.707293 22610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:16.708729 22610 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [term 1 LEADER]: Becoming Leader. State: Replica: a1017ee465104cdd95975ba5a6a7c41d, State: Running, Role: LEADER
I20260810 14:43:16.709219 22610 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1017ee465104cdd95975ba5a6a7c41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } }
I20260810 14:43:16.709419 22598 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:16.712067 22611 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1017ee465104cdd95975ba5a6a7c41d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1017ee465104cdd95975ba5a6a7c41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } } }
I20260810 14:43:16.712165 22612 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1017ee465104cdd95975ba5a6a7c41d. Latest consensus state: current_term: 1 leader_uuid: "a1017ee465104cdd95975ba5a6a7c41d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1017ee465104cdd95975ba5a6a7c41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37225 } } }
I20260810 14:43:16.712590 22612 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:16.712466 22611 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:16.724521 22625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:16.730813 22625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:16.740371 22625 catalog_manager.cc:1347] Generated new cluster ID: 2321ab6cb3194c95b18608b177b7f830
I20260810 14:43:16.740512 22625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:16.765848 22625 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:16.767743 22625 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:16.774960 22625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1017ee465104cdd95975ba5a6a7c41d: Generated new TSK 0
I20260810 14:43:16.775904 22625 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:16.785706 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22503
W20260810 14:43:16.786859 22592 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33117
W20260810 14:43:16.787678 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22503: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:16.787876 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 22503
W20260810 14:43:16.812672 22485 connection.cc:535] client connection to 127.0.0.1:37225 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37225): Connection reset by peer (error 104)
2026-08-10T14:43:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:16 dist-test-slave-h6n0 krb5kdc[22641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:16 dist-test-slave-h6n0 krb5kdc[22641](info): set up 2 sockets
Aug 10 14:43:16 dist-test-slave-h6n0 krb5kdc[22641](info): commencing operation
krb5kdc: starting...
W20260810 14:43:18.182986 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:18 dist-test-slave-h6n0 krb5kdc[22641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:18.569933 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:18.930599 22660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:18.930899 22660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:18.931048 22660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:18.939471 22660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:18.939595 22660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:18.939670 22660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:18.939766 22660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:18.939849 22660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:18.949970 22660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:18.951465 22660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:18.952857 22660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:18.967417 22669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:18.967355 22671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:18.967393 22668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:18.968740 22660 server_base.cc:1034] running on GCE node
I20260810 14:43:18.970074 22660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:18.973074 22660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:18.974516 22660 hybrid_clock.cc:648] HybridClock initialized: now 1786372998974398 us; error 95 us; skew 500 ppm
I20260810 14:43:18.975113 22660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:18 dist-test-slave-h6n0 krb5kdc[22641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786372998, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:18.979700 22660 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:18.981580 22660 webserver.cc:460] Webserver started at http://127.0.0.1:38957/ using document root <none> and password file <none>
I20260810 14:43:18.982324 22660 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:18.982483 22660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:18.982903 22660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:18.986011 22660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "bfba0fb9d66645aebd5eb3b93948cd66"
format_stamp: "Formatted at 2026-08-10 14:43:18 on dist-test-slave-h6n0"
server_key: "9bca49daefa8864ce669aefbdb0ab690"
server_key_iv: "6253e2e2d18974a0ac7242bb0d8315f5"
server_key_version: "encryptionkey@0"
I20260810 14:43:18.986884 22660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "bfba0fb9d66645aebd5eb3b93948cd66"
format_stamp: "Formatted at 2026-08-10 14:43:18 on dist-test-slave-h6n0"
server_key: "9bca49daefa8864ce669aefbdb0ab690"
server_key_iv: "6253e2e2d18974a0ac7242bb0d8315f5"
server_key_version: "encryptionkey@0"
I20260810 14:43:18.992465 22660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:43:18.996110 22682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:18.998086 22660 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260810 14:43:18.998245 22660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "bfba0fb9d66645aebd5eb3b93948cd66"
format_stamp: "Formatted at 2026-08-10 14:43:18 on dist-test-slave-h6n0"
server_key: "9bca49daefa8864ce669aefbdb0ab690"
server_key_iv: "6253e2e2d18974a0ac7242bb0d8315f5"
server_key_version: "encryptionkey@0"
I20260810 14:43:18.998430 22660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:19.004200 22660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:19.008685 22660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:19.008934 22660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:19.029561 22660 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45239
I20260810 14:43:19.029600 22754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45239 every 8 connection(s)
I20260810 14:43:19.031090 22660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:19.034332 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 22660
I20260810 14:43:19.034510 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:19.034955 21725 external_mini_cluster.cc:1428] Setting key b1e063f0c582ac66cc4384d1f1209cba
I20260810 14:43:19.035218 22755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:19.044176 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:19.037128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47723 (local address 127.0.0.1:45239)
0810 14:43:19.038157 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:19.038176 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:19.038364 (+ 188us) server_negotiation.cc:407] Connection header received
0810 14:43:19.040606 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:19.040628 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:19.040966 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:19.041369 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:19.042507 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:19.042516 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:19.042578 (+ 62us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:19.042599 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:19.042669 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:19.042694 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:19.042958 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:19.043109 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 14:43:19.043295 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":421,"threads_started":1}
I20260810 14:43:19.048399 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:19.049729 22755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66: Bootstrap starting.
I20260810 14:43:19.052927 22755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:19.054293 22755 log.cc:824] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:19.055823 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:19.053061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58777 (local address 127.0.0.1:45239)
0810 14:43:19.053220 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:19.053224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:19.053587 (+ 363us) server_negotiation.cc:407] Connection header received
0810 14:43:19.053737 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:19.053741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:19.053789 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:19.053873 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:19.055028 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:19.055036 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:19.055042 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:19.055067 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:19.055130 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:19.055157 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:19.055294 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:19.055387 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 14:43:19.055482 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 14:43:19.057616 22755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66: No bootstrap required, opened a new log
I20260810 14:43:19.064759 22755 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfba0fb9d66645aebd5eb3b93948cd66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } }
I20260810 14:43:19.065129 22755 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:19.065236 22755 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfba0fb9d66645aebd5eb3b93948cd66, State: Initialized, Role: FOLLOWER
I20260810 14:43:19.065845 22755 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfba0fb9d66645aebd5eb3b93948cd66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } }
I20260810 14:43:19.066099 22755 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:19.066242 22755 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:19.066421 22755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:19.068763 22755 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfba0fb9d66645aebd5eb3b93948cd66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } }
I20260810 14:43:19.069247 22755 leader_election.cc:302] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfba0fb9d66645aebd5eb3b93948cd66; no voters:
I20260810 14:43:19.069880 22755 leader_election.cc:288] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:19.070227 22773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:19.070986 22773 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [term 1 LEADER]: Becoming Leader. State: Replica: bfba0fb9d66645aebd5eb3b93948cd66, State: Running, Role: LEADER
I20260810 14:43:19.071508 22773 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfba0fb9d66645aebd5eb3b93948cd66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } }
I20260810 14:43:19.072108 22755 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:19.075364 22777 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfba0fb9d66645aebd5eb3b93948cd66. Latest consensus state: current_term: 1 leader_uuid: "bfba0fb9d66645aebd5eb3b93948cd66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfba0fb9d66645aebd5eb3b93948cd66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } } }
I20260810 14:43:19.075881 22777 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:19.076566 22774 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfba0fb9d66645aebd5eb3b93948cd66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfba0fb9d66645aebd5eb3b93948cd66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45239 } } }
I20260810 14:43:19.076938 22774 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:19.077369 22782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:19.082186 22782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:19.091507 22782 catalog_manager.cc:1347] Generated new cluster ID: ccbbc821396440c79a943446e64292cb
I20260810 14:43:19.091675 22782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:19.114261 22782 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:19.114761 22782 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:19.124058 22782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfba0fb9d66645aebd5eb3b93948cd66: Generated new TSK 0
I20260810 14:43:19.125037 22782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:19.130594 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22660
W20260810 14:43:19.132372 22749 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47723
W20260810 14:43:19.133491 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22660: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:19.133680 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 22660
2026-08-10T14:43:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:19 dist-test-slave-h6n0 krb5kdc[22806](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:19 dist-test-slave-h6n0 krb5kdc[22806](info): set up 2 sockets
Aug 10 14:43:19 dist-test-slave-h6n0 krb5kdc[22806](info): commencing operation
krb5kdc: starting...
W20260810 14:43:20.531388 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:20 dist-test-slave-h6n0 krb5kdc[22806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:20.872145 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:21.264523 22825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:21.264930 22825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:21.265036 22825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:21.273262 22825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:21.273424 22825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:21.273476 22825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:21.273548 22825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:21.273590 22825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:21.285105 22825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:21.286839 22825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:21.288470 22825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:21.303264 22830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:21.303476 22831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:21.303508 22833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:21.304518 22825 server_base.cc:1034] running on GCE node
I20260810 14:43:21.305523 22825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:21.307478 22825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:21.308750 22825 hybrid_clock.cc:648] HybridClock initialized: now 1786373001308684 us; error 46 us; skew 500 ppm
I20260810 14:43:21.309228 22825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:21 dist-test-slave-h6n0 krb5kdc[22806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373001, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:21.317155 22825 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:21.319397 22825 webserver.cc:460] Webserver started at http://127.0.0.1:35627/ using document root <none> and password file <none>
I20260810 14:43:21.320309 22825 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:21.320470 22825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:21.320881 22825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:21.324673 22825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "92aa9adea7f546ea927451bb511aad39"
format_stamp: "Formatted at 2026-08-10 14:43:21 on dist-test-slave-h6n0"
server_key: "0f392b277e6a85e857fd43bab06fc95e"
server_key_iv: "6446be095a6789e8eb9e9ebc6c8c3335"
server_key_version: "encryptionkey@0"
I20260810 14:43:21.325662 22825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "92aa9adea7f546ea927451bb511aad39"
format_stamp: "Formatted at 2026-08-10 14:43:21 on dist-test-slave-h6n0"
server_key: "0f392b277e6a85e857fd43bab06fc95e"
server_key_iv: "6446be095a6789e8eb9e9ebc6c8c3335"
server_key_version: "encryptionkey@0"
I20260810 14:43:21.332252 22825 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 14:43:21.337018 22849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:21.339048 22825 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:43:21.339298 22825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "92aa9adea7f546ea927451bb511aad39"
format_stamp: "Formatted at 2026-08-10 14:43:21 on dist-test-slave-h6n0"
server_key: "0f392b277e6a85e857fd43bab06fc95e"
server_key_iv: "6446be095a6789e8eb9e9ebc6c8c3335"
server_key_version: "encryptionkey@0"
I20260810 14:43:21.339550 22825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:21.364765 22825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:21.371127 22825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:21.371497 22825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:21.394457 22825 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45531
I20260810 14:43:21.394599 22931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45531 every 8 connection(s)
I20260810 14:43:21.396248 22825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:21.397557 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 22825
I20260810 14:43:21.397784 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:21.398361 21725 external_mini_cluster.cc:1428] Setting key 2513010d5440afc27dd769909a45e374
I20260810 14:43:21.401567 22932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:21.415373 22932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39: Bootstrap starting.
Aug 10 14:43:21 dist-test-slave-h6n0 krb5kdc[22806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 14:43:21.418510 22932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:21.419682 22932 log.cc:824] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:21.422607 22932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39: No bootstrap required, opened a new log
I20260810 14:43:21.426213 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:21.400623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58329 (local address 127.0.0.1:45531)
0810 14:43:21.401597 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:21.401611 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:21.401700 (+ 89us) server_negotiation.cc:407] Connection header received
0810 14:43:21.403700 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:21.403726 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:21.404217 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:21.404624 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:21.406495 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:21.408160 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:21.410141 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:21.411097 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:21.411286 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:21.416996 (+ 5710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:21.417009 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:21.417029 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:21.417080 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:21.420979 (+ 3899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:21.421867 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:21.421875 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:21.421883 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:21.422078 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:21.422573 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:21.422578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:21.422580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:21.423360 (+ 780us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:21.423740 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:21.424273 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 14:43:21.424802 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":446,"threads_started":1}
I20260810 14:43:21.431423 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:21.431957 22932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92aa9adea7f546ea927451bb511aad39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } }
I20260810 14:43:21.432343 22932 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:21.432451 22932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92aa9adea7f546ea927451bb511aad39, State: Initialized, Role: FOLLOWER
I20260810 14:43:21.433033 22932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92aa9adea7f546ea927451bb511aad39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } }
I20260810 14:43:21.433400 22932 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:21.433624 22932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:21.433849 22932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:21.437079 22932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92aa9adea7f546ea927451bb511aad39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } }
I20260810 14:43:21.437908 22932 leader_election.cc:302] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92aa9adea7f546ea927451bb511aad39; no voters:
I20260810 14:43:21.438777 22932 leader_election.cc:288] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:21.438898 22949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:21.440240 22949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [term 1 LEADER]: Becoming Leader. State: Replica: 92aa9adea7f546ea927451bb511aad39, State: Running, Role: LEADER
I20260810 14:43:21.441025 22949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92aa9adea7f546ea927451bb511aad39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } }
I20260810 14:43:21.441721 22932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:21.444240 22951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92aa9adea7f546ea927451bb511aad39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92aa9adea7f546ea927451bb511aad39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } } }
I20260810 14:43:21.444697 22951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:21.449038 22956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92aa9adea7f546ea927451bb511aad39. Latest consensus state: current_term: 1 leader_uuid: "92aa9adea7f546ea927451bb511aad39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92aa9adea7f546ea927451bb511aad39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45531 } } }
I20260810 14:43:21.449632 22956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:21.451232 22961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:21.460386 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:21.437196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49499 (local address 127.0.0.1:45531)
0810 14:43:21.437424 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:21.437431 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:21.437884 (+ 453us) server_negotiation.cc:407] Connection header received
0810 14:43:21.438045 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:21.438050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:21.438150 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:21.438275 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:21.440643 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:21.441921 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:21.444067 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:21.445164 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:21.445372 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:21.450101 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:21.450109 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:21.450115 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:21.450167 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:21.455521 (+ 5354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:21.456748 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:21.456755 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:21.456759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:21.457024 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:21.457600 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:21.457606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:21.457609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:21.458204 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:21.458561 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:21.459591 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0810 14:43:21.459913 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 14:43:21.463413 22961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:21.479297 22961 catalog_manager.cc:1347] Generated new cluster ID: 77ca5ef7b386479e9f60dba7a9d15325
I20260810 14:43:21.479445 22961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:21.501621 22961 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:21.503252 22961 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:21.509784 22961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92aa9adea7f546ea927451bb511aad39: Generated new TSK 0
I20260810 14:43:21.510952 22961 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:21.524601 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 22825
I20260810 14:43:21.525964 22927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:21.626017 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 22825
2026-08-10T14:43:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:21 dist-test-slave-h6n0 krb5kdc[22984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:21 dist-test-slave-h6n0 krb5kdc[22984](info): set up 2 sockets
Aug 10 14:43:21 dist-test-slave-h6n0 krb5kdc[22984](info): commencing operation
krb5kdc: starting...
W20260810 14:43:23.078990 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.400s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:23 dist-test-slave-h6n0 krb5kdc[22984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:23.435240 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42181
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:23.799861 23001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:23.800153 23001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:23.800243 23001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:23.807216 23001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:23.807318 23001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:23.807363 23001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:23.807431 23001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:23.807475 23001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:23.817293 23001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:23.818477 23001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:23.819828 23001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:23.832422 23008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:23.832623 23015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:23.832608 23009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:23.833529 23001 server_base.cc:1034] running on GCE node
I20260810 14:43:23.834323 23001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:23.835752 23001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:23.836979 23001 hybrid_clock.cc:648] HybridClock initialized: now 1786373003836920 us; error 41 us; skew 500 ppm
I20260810 14:43:23.837306 23001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:23 dist-test-slave-h6n0 krb5kdc[22984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:23.840689 23001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:23.842193 23001 webserver.cc:460] Webserver started at http://127.0.0.1:36605/ using document root <none> and password file <none>
I20260810 14:43:23.842916 23001 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:23.843036 23001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:23.843293 23001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:23.846326 23001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "006556e8ad19478ebeadbb3f85da357c"
format_stamp: "Formatted at 2026-08-10 14:43:23 on dist-test-slave-h6n0"
server_key: "87dabc577ebed688dcf46f5226146701"
server_key_iv: "e208f5407c1dc40e8dcd556165e83590"
server_key_version: "encryptionkey@0"
I20260810 14:43:23.847143 23001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "006556e8ad19478ebeadbb3f85da357c"
format_stamp: "Formatted at 2026-08-10 14:43:23 on dist-test-slave-h6n0"
server_key: "87dabc577ebed688dcf46f5226146701"
server_key_iv: "e208f5407c1dc40e8dcd556165e83590"
server_key_version: "encryptionkey@0"
I20260810 14:43:23.852136 23001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:43:23.855935 23022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:23.857416 23001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:43:23.857578 23001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "006556e8ad19478ebeadbb3f85da357c"
format_stamp: "Formatted at 2026-08-10 14:43:23 on dist-test-slave-h6n0"
server_key: "87dabc577ebed688dcf46f5226146701"
server_key_iv: "e208f5407c1dc40e8dcd556165e83590"
server_key_version: "encryptionkey@0"
I20260810 14:43:23.857764 23001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:23.876789 23001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:23.881024 23001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:23.881260 23001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:23.907382 23001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42181
I20260810 14:43:23.907411 23095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42181 every 8 connection(s)
I20260810 14:43:23.908843 23001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:23.910972 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23001
I20260810 14:43:23.911164 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:23.911661 21725 external_mini_cluster.cc:1428] Setting key adf0967d5494fca2f6de45780c3e4d2b
I20260810 14:43:23.915295 23096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:23.928855 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:23.913760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48943 (local address 127.0.0.1:42181)
0810 14:43:23.914893 (+ 1133us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:23.914913 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:23.914983 (+ 70us) server_negotiation.cc:407] Connection header received
0810 14:43:23.917861 (+ 2878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:23.917894 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:23.918474 (+ 580us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:23.918961 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:23.919526 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:23.921559 (+ 2033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:23.922983 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:23.923992 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:23.924179 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:23.925862 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:23.925878 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:23.925898 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:23.926666 (+ 768us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:23.926780 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:23.926811 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:23.926952 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:23.927224 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 14:43:23.927516 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":430,"threads_started":1}
I20260810 14:43:23.930224 23096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c: Bootstrap starting.
I20260810 14:43:23.932737 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:23.933513 23096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:23.934504 23096 log.cc:824] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:23.939330 23096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c: No bootstrap required, opened a new log
I20260810 14:43:23.946605 23101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:23.937633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42309 (local address 127.0.0.1:42181)
0810 14:43:23.937983 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:23.937991 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:23.938240 (+ 249us) server_negotiation.cc:407] Connection header received
0810 14:43:23.938393 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:23.938400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:23.938522 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:23.938851 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:23.939721 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:23.941134 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:23.942707 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:23.943623 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:23.943844 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:23.945026 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:23.945036 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:23.945043 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:23.945075 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:23.945143 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:23.945168 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:23.945302 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:23.945764 (+ 462us) server_negotiation.cc:299] Negotiation successful
0810 14:43:23.945926 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260810 14:43:23.951098 23096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006556e8ad19478ebeadbb3f85da357c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } }
I20260810 14:43:23.951493 23096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:23.951638 23096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 006556e8ad19478ebeadbb3f85da357c, State: Initialized, Role: FOLLOWER
I20260810 14:43:23.952230 23096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006556e8ad19478ebeadbb3f85da357c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } }
I20260810 14:43:23.952498 23096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:23.952670 23096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:23.952862 23096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:23.955250 23096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006556e8ad19478ebeadbb3f85da357c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } }
I20260810 14:43:23.955793 23096 leader_election.cc:302] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 006556e8ad19478ebeadbb3f85da357c; no voters:
I20260810 14:43:23.956667 23096 leader_election.cc:288] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:23.956708 23112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:23.957756 23112 raft_consensus.cc:686] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [term 1 LEADER]: Becoming Leader. State: Replica: 006556e8ad19478ebeadbb3f85da357c, State: Running, Role: LEADER
I20260810 14:43:23.958448 23112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006556e8ad19478ebeadbb3f85da357c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } }
I20260810 14:43:23.960211 23096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:23.962440 23114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "006556e8ad19478ebeadbb3f85da357c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006556e8ad19478ebeadbb3f85da357c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } } }
I20260810 14:43:23.962931 23114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:23.963711 23116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 006556e8ad19478ebeadbb3f85da357c. Latest consensus state: current_term: 1 leader_uuid: "006556e8ad19478ebeadbb3f85da357c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006556e8ad19478ebeadbb3f85da357c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42181 } } }
I20260810 14:43:23.964677 23116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:23.967132 23128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:23.971944 23128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:23.982179 23128 catalog_manager.cc:1347] Generated new cluster ID: 0ffa3aa68c01495ba0096e95f655b6a1
I20260810 14:43:23.982313 23128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:24.009229 23128 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:24.011081 23128 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:24.020444 23128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 006556e8ad19478ebeadbb3f85da357c: Generated new TSK 0
I20260810 14:43:24.021481 23128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:24.032994 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23001
W20260810 14:43:24.034291 23091 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48943
W20260810 14:43:24.035058 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23001: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:24.035210 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23001
2026-08-10T14:43:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:24 dist-test-slave-h6n0 krb5kdc[23145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:24 dist-test-slave-h6n0 krb5kdc[23145](info): set up 2 sockets
Aug 10 14:43:24 dist-test-slave-h6n0 krb5kdc[23145](info): commencing operation
krb5kdc: starting...
W20260810 14:43:25.462280 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.383s user 0.007s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:25 dist-test-slave-h6n0 krb5kdc[23145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:25.822567 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37379
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:26.203490 23163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:26.203941 23163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:26.204054 23163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:26.214250 23163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:26.214462 23163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:26.214602 23163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:26.214735 23163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:26.214787 23163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:26.227454 23163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:26.229292 23163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:26.230988 23163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:26.247160 23179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:26.247328 23172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:26.247272 23174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:26.248952 23163 server_base.cc:1034] running on GCE node
I20260810 14:43:26.250137 23163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:26.252277 23163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:26.253652 23163 hybrid_clock.cc:648] HybridClock initialized: now 1786373006253613 us; error 154 us; skew 500 ppm
I20260810 14:43:26.254123 23163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:26 dist-test-slave-h6n0 krb5kdc[23145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373006, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:26.257827 23163 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:26.259624 23163 webserver.cc:460] Webserver started at http://127.0.0.1:43131/ using document root <none> and password file <none>
I20260810 14:43:26.260452 23163 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:26.260586 23163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:26.260851 23163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:26.264068 23163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "81612fbc173342f68fba785e6e2ee18b"
format_stamp: "Formatted at 2026-08-10 14:43:26 on dist-test-slave-h6n0"
server_key: "7ca38c88857aec0eee7f596014cbb91d"
server_key_iv: "06b81a79db62b5b28537681521065bf7"
server_key_version: "encryptionkey@0"
I20260810 14:43:26.264837 23163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "81612fbc173342f68fba785e6e2ee18b"
format_stamp: "Formatted at 2026-08-10 14:43:26 on dist-test-slave-h6n0"
server_key: "7ca38c88857aec0eee7f596014cbb91d"
server_key_iv: "06b81a79db62b5b28537681521065bf7"
server_key_version: "encryptionkey@0"
I20260810 14:43:26.271139 23163 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 14:43:26.276726 23188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:26.278748 23163 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:43:26.278921 23163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "81612fbc173342f68fba785e6e2ee18b"
format_stamp: "Formatted at 2026-08-10 14:43:26 on dist-test-slave-h6n0"
server_key: "7ca38c88857aec0eee7f596014cbb91d"
server_key_iv: "06b81a79db62b5b28537681521065bf7"
server_key_version: "encryptionkey@0"
I20260810 14:43:26.279210 23163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:26.286446 23163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:26.292236 23163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:26.292582 23163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:26.318320 23274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37379 every 8 connection(s)
I20260810 14:43:26.318393 23163 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37379
I20260810 14:43:26.320108 23163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:26.324276 23275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:26.329109 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23163
I20260810 14:43:26.329347 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:26.329804 21725 external_mini_cluster.cc:1428] Setting key 5689a6a2af50c624c455734a3ee19337
I20260810 14:43:26.336242 23275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b: Bootstrap starting.
I20260810 14:43:26.340003 23275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:26.341089 23275 log.cc:824] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:26.345494 23275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b: No bootstrap required, opened a new log
I20260810 14:43:26.346248 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:26.332754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55189 (local address 127.0.0.1:37379)
0810 14:43:26.334193 (+ 1439us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:26.334325 (+ 132us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:26.334406 (+ 81us) server_negotiation.cc:407] Connection header received
0810 14:43:26.337092 (+ 2686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:26.337134 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:26.337698 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:26.338224 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:26.343779 (+ 5555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:26.343789 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:26.343812 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:26.343844 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:26.343952 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:26.344002 (+ 50us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:26.344258 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:26.344483 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 14:43:26.344752 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":663,"threads_started":1}
I20260810 14:43:26.351094 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:26.356259 23275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81612fbc173342f68fba785e6e2ee18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37379 } }
I20260810 14:43:26.356941 23275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:26.357076 23275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81612fbc173342f68fba785e6e2ee18b, State: Initialized, Role: FOLLOWER
I20260810 14:43:26.357825 23275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81612fbc173342f68fba785e6e2ee18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37379 } }
I20260810 14:43:26.358124 23275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:26.358307 23275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:26.358589 23275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:26.360724 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:26.356559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44769 (local address 127.0.0.1:37379)
0810 14:43:26.356809 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:26.356820 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:26.357275 (+ 455us) server_negotiation.cc:407] Connection header received
0810 14:43:26.357546 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:26.357551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:26.357621 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:26.357744 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:26.359361 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:26.359466 (+ 105us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:26.359475 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:26.359507 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:26.359566 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:26.359591 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:26.360121 (+ 530us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:26.360189 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 14:43:26.360274 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 14:43:26.362968 23275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81612fbc173342f68fba785e6e2ee18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37379 } }
I20260810 14:43:26.363567 23275 leader_election.cc:302] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81612fbc173342f68fba785e6e2ee18b; no voters:
I20260810 14:43:26.364262 23275 leader_election.cc:288] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:26.364367 23290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:26.365134 23290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [term 1 LEADER]: Becoming Leader. State: Replica: 81612fbc173342f68fba785e6e2ee18b, State: Running, Role: LEADER
I20260810 14:43:26.365854 23290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81612fbc173342f68fba785e6e2ee18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37379 } }
I20260810 14:43:26.366695 23275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:26.369274 23296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81612fbc173342f68fba785e6e2ee18b. Latest consensus state: current_term: 1 leader_uuid: "81612fbc173342f68fba785e6e2ee18b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81612fbc173342f68fba785e6e2ee18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37379 } } }
I20260810 14:43:26.369819 23296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:26.370260 23291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81612fbc173342f68fba785e6e2ee18b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81612fbc173342f68fba785e6e2ee18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37379 } } }
I20260810 14:43:26.371317 23291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:26.373224 23303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:26.382658 23303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:26.394291 23303 catalog_manager.cc:1347] Generated new cluster ID: 45ae3f5a78ef4e5494ea3798073e2507
I20260810 14:43:26.394482 23303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:26.413096 23303 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:26.413663 23303 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:26.422830 23303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81612fbc173342f68fba785e6e2ee18b: Generated new TSK 0
I20260810 14:43:26.423933 23303 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:26.432071 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23163
W20260810 14:43:26.433847 23267 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55189
W20260810 14:43:26.435112 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23163: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:26.435303 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23163
2026-08-10T14:43:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:26 dist-test-slave-h6n0 krb5kdc[23323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:26 dist-test-slave-h6n0 krb5kdc[23323](info): set up 2 sockets
Aug 10 14:43:26 dist-test-slave-h6n0 krb5kdc[23323](info): commencing operation
krb5kdc: starting...
W20260810 14:43:27.873900 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.389s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:28 dist-test-slave-h6n0 krb5kdc[23323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:28.237494 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33007
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:28.694087 23342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:28.694602 23342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:28.694792 23342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:28.703616 23342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:28.703828 23342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:28.703948 23342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:28.704061 23342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:28.704149 23342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:28.715745 23342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:28.717592 23342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:28.719395 23342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:28.734725 23347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:28.734822 23352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:28.734685 23349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:28.736572 23342 server_base.cc:1034] running on GCE node
I20260810 14:43:28.737888 23342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:28.740250 23342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:28.741617 23342 hybrid_clock.cc:648] HybridClock initialized: now 1786373008741503 us; error 136 us; skew 500 ppm
I20260810 14:43:28.742274 23342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:28 dist-test-slave-h6n0 krb5kdc[23323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373008, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:28.747119 23342 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:28.749270 23342 webserver.cc:460] Webserver started at http://127.0.0.1:43181/ using document root <none> and password file <none>
I20260810 14:43:28.750217 23342 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:28.750365 23342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:28.750797 23342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:28.755590 23342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "4b2b937189744a9084b2eea6ba0312a1"
format_stamp: "Formatted at 2026-08-10 14:43:28 on dist-test-slave-h6n0"
server_key: "13ded5499f854e7b1e50bc546ce152f5"
server_key_iv: "b1538c698bd1a7488b6d714de3253808"
server_key_version: "encryptionkey@0"
I20260810 14:43:28.756672 23342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "4b2b937189744a9084b2eea6ba0312a1"
format_stamp: "Formatted at 2026-08-10 14:43:28 on dist-test-slave-h6n0"
server_key: "13ded5499f854e7b1e50bc546ce152f5"
server_key_iv: "b1538c698bd1a7488b6d714de3253808"
server_key_version: "encryptionkey@0"
I20260810 14:43:28.764631 23342 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260810 14:43:28.769640 23364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:28.771754 23342 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 14:43:28.772029 23342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "4b2b937189744a9084b2eea6ba0312a1"
format_stamp: "Formatted at 2026-08-10 14:43:28 on dist-test-slave-h6n0"
server_key: "13ded5499f854e7b1e50bc546ce152f5"
server_key_iv: "b1538c698bd1a7488b6d714de3253808"
server_key_version: "encryptionkey@0"
I20260810 14:43:28.772287 23342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:28.805826 23342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:28.811686 23342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:28.812078 23342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:28.838701 23342 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33007
I20260810 14:43:28.838745 23439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33007 every 8 connection(s)
I20260810 14:43:28.840587 23342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:28.845273 23441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:28.845317 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23342
I20260810 14:43:28.845494 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:28.846110 21725 external_mini_cluster.cc:1428] Setting key 39f4ff63b5af6451347a967e46cb78df
I20260810 14:43:28.862586 23441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1: Bootstrap starting.
Aug 10 14:43:28 dist-test-slave-h6n0 krb5kdc[23323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 14:43:28.867394 23441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:28.868703 23441 log.cc:824] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:28.873613 23441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1: No bootstrap required, opened a new log
I20260810 14:43:28.879832 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:28.848888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:34257 (local address 127.0.0.1:33007)
0810 14:43:28.849996 (+ 1108us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:28.850013 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:28.850075 (+ 62us) server_negotiation.cc:407] Connection header received
0810 14:43:28.852989 (+ 2914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:28.853030 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:28.853648 (+ 618us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:28.854219 (+ 571us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:28.856441 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:28.858213 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:28.860407 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:28.861457 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:28.861683 (+ 226us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:28.868001 (+ 6318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:28.868014 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:28.868041 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:28.868174 (+ 133us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:28.873141 (+ 4967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:28.874696 (+ 1555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:28.874707 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:28.874717 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:28.874961 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:28.875777 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:28.875783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:28.875788 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:28.876726 (+ 938us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:28.877117 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:28.878030 (+ 913us) server_negotiation.cc:299] Negotiation successful
0810 14:43:28.878505 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":523,"threads_started":1}
I20260810 14:43:28.885789 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:28.886341 23441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2b937189744a9084b2eea6ba0312a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33007 } }
I20260810 14:43:28.887163 23441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:28.887300 23441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b2b937189744a9084b2eea6ba0312a1, State: Initialized, Role: FOLLOWER
I20260810 14:43:28.888253 23441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2b937189744a9084b2eea6ba0312a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33007 } }
I20260810 14:43:28.888655 23441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:28.888916 23441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:28.889192 23441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:28.893091 23441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2b937189744a9084b2eea6ba0312a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33007 } }
I20260810 14:43:28.894668 23441 leader_election.cc:302] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b2b937189744a9084b2eea6ba0312a1; no voters:
I20260810 14:43:28.895710 23441 leader_election.cc:288] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:28.895884 23457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:28.898968 23457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [term 1 LEADER]: Becoming Leader. State: Replica: 4b2b937189744a9084b2eea6ba0312a1, State: Running, Role: LEADER
I20260810 14:43:28.899777 23441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:28.900940 23457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2b937189744a9084b2eea6ba0312a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33007 } }
I20260810 14:43:28.914953 23459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b2b937189744a9084b2eea6ba0312a1. Latest consensus state: current_term: 1 leader_uuid: "4b2b937189744a9084b2eea6ba0312a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2b937189744a9084b2eea6ba0312a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33007 } } }
I20260810 14:43:28.915506 23459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:28.914460 23458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b2b937189744a9084b2eea6ba0312a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b2b937189744a9084b2eea6ba0312a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33007 } } }
I20260810 14:43:28.916518 23458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:28.926160 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:28.897950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:57729 (local address 127.0.0.1:33007)
0810 14:43:28.898158 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:28.898165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:28.898188 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:43:28.898503 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:28.898509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:28.898627 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:28.898718 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:28.901166 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:28.902740 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:28.905519 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:28.906509 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:28.906667 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:28.911811 (+ 5144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:28.911823 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:28.911831 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:28.911892 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:28.919496 (+ 7604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:28.921147 (+ 1651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:28.921156 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:28.921162 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:28.921364 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:28.922116 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:28.922124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:28.922130 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:28.922833 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:28.923216 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:28.924919 (+ 1703us) server_negotiation.cc:299] Negotiation successful
0810 14:43:28.925236 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 14:43:28.932375 23468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:28.940723 23468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:28.954910 23468 catalog_manager.cc:1347] Generated new cluster ID: 87265a2e57b642be8d3ad8f6f10ac8df
I20260810 14:43:28.955101 23468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:28.987798 23468 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:28.989796 23468 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:29.004313 23468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b2b937189744a9084b2eea6ba0312a1: Generated new TSK 0
I20260810 14:43:29.005604 23468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:29.016711 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23342
I20260810 14:43:29.018059 23435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:29.124557 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23342
W20260810 14:43:29.158476 23319 connection.cc:535] client connection to 127.0.0.1:33007 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33007): Connection reset by peer (error 104)
2026-08-10T14:43:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2695 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:29 dist-test-slave-h6n0 krb5kdc[23493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:29 dist-test-slave-h6n0 krb5kdc[23493](info): set up 2 sockets
Aug 10 14:43:29 dist-test-slave-h6n0 krb5kdc[23493](info): commencing operation
krb5kdc: starting...
W20260810 14:43:30.569765 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.390s user 0.002s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:30 dist-test-slave-h6n0 krb5kdc[23493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:30.934865 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35741
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:31.337913 23509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:31.338225 23509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:31.338321 23509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:31.345582 23509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:31.345695 23509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:31.345742 23509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:31.345796 23509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:31.345841 23509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:31.355593 23509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:31.356808 23509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:31.358071 23509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:31.372059 23522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:31.372211 23519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:31.372176 23520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:31.373415 23509 server_base.cc:1034] running on GCE node
I20260810 14:43:31.374320 23509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:31.375917 23509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:31.377153 23509 hybrid_clock.cc:648] HybridClock initialized: now 1786373011377104 us; error 40 us; skew 500 ppm
I20260810 14:43:31.377579 23509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:31 dist-test-slave-h6n0 krb5kdc[23493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373011, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:31.385378 23509 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:31.387820 23509 webserver.cc:460] Webserver started at http://127.0.0.1:43515/ using document root <none> and password file <none>
I20260810 14:43:31.388738 23509 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:31.388923 23509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:31.389288 23509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:31.392853 23509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "cca349a3959648549994fe3121a24220"
format_stamp: "Formatted at 2026-08-10 14:43:31 on dist-test-slave-h6n0"
server_key: "d9d00a6ad48a5b1fc383909c5155b90e"
server_key_iv: "a751bc71d6e53d7ea4bf761cc08e0344"
server_key_version: "encryptionkey@0"
I20260810 14:43:31.393754 23509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "cca349a3959648549994fe3121a24220"
format_stamp: "Formatted at 2026-08-10 14:43:31 on dist-test-slave-h6n0"
server_key: "d9d00a6ad48a5b1fc383909c5155b90e"
server_key_iv: "a751bc71d6e53d7ea4bf761cc08e0344"
server_key_version: "encryptionkey@0"
I20260810 14:43:31.400215 23509 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 14:43:31.404274 23531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:31.405680 23509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:43:31.405893 23509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "cca349a3959648549994fe3121a24220"
format_stamp: "Formatted at 2026-08-10 14:43:31 on dist-test-slave-h6n0"
server_key: "d9d00a6ad48a5b1fc383909c5155b90e"
server_key_iv: "a751bc71d6e53d7ea4bf761cc08e0344"
server_key_version: "encryptionkey@0"
I20260810 14:43:31.406131 23509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:31.420773 23509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:31.426159 23509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:31.426580 23509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:31.449607 23509 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35741
I20260810 14:43:31.449714 23611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35741 every 8 connection(s)
I20260810 14:43:31.451228 23509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:31.453446 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23509
I20260810 14:43:31.453658 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:31.454288 21725 external_mini_cluster.cc:1428] Setting key f3fa2040fea07135e9a9bab67b7f9324
I20260810 14:43:31.456365 23613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:31.468189 23616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:31.456362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:41581 (local address 127.0.0.1:35741)
0810 14:43:31.457132 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:31.457144 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:31.457185 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:43:31.459514 (+ 2329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:31.459536 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:31.459922 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:31.460324 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:31.460877 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:31.462320 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:31.463667 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:31.464311 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:31.464517 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:31.465826 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:31.465833 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:31.465845 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:31.466393 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:31.466481 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:31.466518 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:31.466764 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:31.467047 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 14:43:31.467208 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":337,"threads_started":1}
I20260810 14:43:31.470613 23613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220: Bootstrap starting.
I20260810 14:43:31.472394 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:31.473856 23613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:31.474984 23613 log.cc:824] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:31.478931 23613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220: No bootstrap required, opened a new log
I20260810 14:43:31.485296 23616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:31.477772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:57871 (local address 127.0.0.1:35741)
0810 14:43:31.477978 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:31.477988 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:31.478610 (+ 622us) server_negotiation.cc:407] Connection header received
0810 14:43:31.479072 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:31.479080 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:31.479171 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:31.479293 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:31.479924 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:31.480809 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:31.482372 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:31.483023 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:31.483212 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:31.484517 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:31.484522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:31.484526 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:31.484553 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:31.484593 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:31.484610 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:31.484751 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:31.484958 (+ 207us) server_negotiation.cc:299] Negotiation successful
0810 14:43:31.485018 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 14:43:31.487583 23613 raft_consensus.cc:348] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cca349a3959648549994fe3121a24220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35741 } }
I20260810 14:43:31.488010 23613 raft_consensus.cc:374] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:31.488153 23613 raft_consensus.cc:729] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cca349a3959648549994fe3121a24220, State: Initialized, Role: FOLLOWER
I20260810 14:43:31.488763 23613 consensus_queue.cc:260] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cca349a3959648549994fe3121a24220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35741 } }
I20260810 14:43:31.489048 23613 raft_consensus.cc:388] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:31.489225 23613 raft_consensus.cc:482] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:31.489418 23613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:31.492282 23613 raft_consensus.cc:504] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cca349a3959648549994fe3121a24220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35741 } }
I20260810 14:43:31.492911 23613 leader_election.cc:302] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cca349a3959648549994fe3121a24220; no voters:
I20260810 14:43:31.493505 23613 leader_election.cc:288] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:31.493665 23625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:31.494467 23625 raft_consensus.cc:686] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [term 1 LEADER]: Becoming Leader. State: Replica: cca349a3959648549994fe3121a24220, State: Running, Role: LEADER
I20260810 14:43:31.495098 23625 consensus_queue.cc:237] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cca349a3959648549994fe3121a24220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35741 } }
I20260810 14:43:31.496112 23613 sys_catalog.cc:564] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:31.497646 23628 sys_catalog.cc:455] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cca349a3959648549994fe3121a24220. Latest consensus state: current_term: 1 leader_uuid: "cca349a3959648549994fe3121a24220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cca349a3959648549994fe3121a24220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35741 } } }
I20260810 14:43:31.498158 23628 sys_catalog.cc:458] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:31.497939 23627 sys_catalog.cc:455] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cca349a3959648549994fe3121a24220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cca349a3959648549994fe3121a24220" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35741 } } }
I20260810 14:43:31.498354 23627 sys_catalog.cc:458] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:31.500945 23635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:31.507261 23635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:31.515906 23635 catalog_manager.cc:1347] Generated new cluster ID: 568ebb451d604c90ada55799bfa51318
I20260810 14:43:31.516023 23635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:31.529595 23635 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:31.531414 23635 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:31.543529 23635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cca349a3959648549994fe3121a24220: Generated new TSK 0
I20260810 14:43:31.544541 23635 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:31.562855 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23509
W20260810 14:43:31.564432 23604 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.52:41581
W20260810 14:43:31.565359 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23509: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:31.565557 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23509
W20260810 14:43:31.593529 23490 connection.cc:535] client connection to 127.0.0.1:35741 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35741): Connection reset by peer (error 104)
2026-08-10T14:43:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2434 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:31 dist-test-slave-h6n0 krb5kdc[23661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:31 dist-test-slave-h6n0 krb5kdc[23661](info): set up 2 sockets
Aug 10 14:43:31 dist-test-slave-h6n0 krb5kdc[23661](info): commencing operation
krb5kdc: starting...
W20260810 14:43:33.010407 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.395s user 0.003s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:33 dist-test-slave-h6n0 krb5kdc[23661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:33.374765 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44827
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:33.745007 23682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:33.745314 23682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:33.745471 23682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:33.752681 23682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:33.752789 23682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:33.752888 23682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:33.753005 23682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:33.753073 23682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:33.762837 23682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:33.764197 23682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:33.765499 23682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:33.781268 23691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:33.781467 23692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:33.781489 23696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:33.783528 23682 server_base.cc:1034] running on GCE node
I20260810 14:43:33.784773 23682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:33.786640 23682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:33.787882 23682 hybrid_clock.cc:648] HybridClock initialized: now 1786373013787827 us; error 42 us; skew 500 ppm
I20260810 14:43:33.788347 23682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:33 dist-test-slave-h6n0 krb5kdc[23661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373013, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:33.792223 23682 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:33.793880 23682 webserver.cc:460] Webserver started at http://127.0.0.1:40799/ using document root <none> and password file <none>
I20260810 14:43:33.794670 23682 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:33.794811 23682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:33.795140 23682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:33.798313 23682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "0af0fa8676a04a028a090223f5a181b4"
format_stamp: "Formatted at 2026-08-10 14:43:33 on dist-test-slave-h6n0"
server_key: "39108adf77d6529c3a549e86faff9af0"
server_key_iv: "c58201d7faa99655183f229d1e0d31d3"
server_key_version: "encryptionkey@0"
I20260810 14:43:33.799193 23682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "0af0fa8676a04a028a090223f5a181b4"
format_stamp: "Formatted at 2026-08-10 14:43:33 on dist-test-slave-h6n0"
server_key: "39108adf77d6529c3a549e86faff9af0"
server_key_iv: "c58201d7faa99655183f229d1e0d31d3"
server_key_version: "encryptionkey@0"
I20260810 14:43:33.805260 23682 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 14:43:33.810263 23706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:33.811915 23682 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 14:43:33.812114 23682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "0af0fa8676a04a028a090223f5a181b4"
format_stamp: "Formatted at 2026-08-10 14:43:33 on dist-test-slave-h6n0"
server_key: "39108adf77d6529c3a549e86faff9af0"
server_key_iv: "c58201d7faa99655183f229d1e0d31d3"
server_key_version: "encryptionkey@0"
I20260810 14:43:33.812323 23682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:33.818945 23682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:33.823498 23682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:33.823827 23682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:33.846473 23682 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44827
I20260810 14:43:33.846480 23772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44827 every 8 connection(s)
I20260810 14:43:33.848206 23682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:33.848857 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23682
I20260810 14:43:33.849115 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:33.849833 21725 external_mini_cluster.cc:1428] Setting key 133aa0f55dfc78b6107eb4acd0d5b0da
I20260810 14:43:33.853019 23773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:33.859630 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:33.851897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:57987 (local address 127.0.0.1:44827)
0810 14:43:33.853101 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:33.853174 (+ 73us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:33.853261 (+ 87us) server_negotiation.cc:407] Connection header received
0810 14:43:33.855526 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:33.855553 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:33.855947 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:33.856426 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:33.857650 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:33.857657 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:33.857670 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:33.857685 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:33.857765 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:33.857792 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:33.858086 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:33.858259 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 14:43:33.858666 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":486,"threads_started":1}
I20260810 14:43:33.864774 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:33.865427 23773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4: Bootstrap starting.
I20260810 14:43:33.868870 23773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:33.870551 23773 log.cc:824] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:33.873111 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:33.870477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:38951 (local address 127.0.0.1:44827)
0810 14:43:33.870710 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:33.870717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:33.870866 (+ 149us) server_negotiation.cc:407] Connection header received
0810 14:43:33.871129 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:33.871133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:33.871182 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:33.871311 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:33.872422 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:33.872435 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:33.872440 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:33.872457 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:33.872504 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:33.872519 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:33.872647 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:33.872750 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 14:43:33.872815 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260810 14:43:33.873695 23773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4: No bootstrap required, opened a new log
I20260810 14:43:33.884178 23773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af0fa8676a04a028a090223f5a181b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260810 14:43:33.884660 23773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:33.884775 23773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af0fa8676a04a028a090223f5a181b4, State: Initialized, Role: FOLLOWER
I20260810 14:43:33.885532 23773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af0fa8676a04a028a090223f5a181b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260810 14:43:33.885807 23773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:33.886009 23773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:33.886225 23773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:33.889031 23773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af0fa8676a04a028a090223f5a181b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260810 14:43:33.889643 23773 leader_election.cc:302] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0af0fa8676a04a028a090223f5a181b4; no voters:
I20260810 14:43:33.890434 23773 leader_election.cc:288] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:33.890661 23786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:33.891420 23786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [term 1 LEADER]: Becoming Leader. State: Replica: 0af0fa8676a04a028a090223f5a181b4, State: Running, Role: LEADER
I20260810 14:43:33.892059 23786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af0fa8676a04a028a090223f5a181b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } }
I20260810 14:43:33.892917 23773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:33.895023 23789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0af0fa8676a04a028a090223f5a181b4. Latest consensus state: current_term: 1 leader_uuid: "0af0fa8676a04a028a090223f5a181b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af0fa8676a04a028a090223f5a181b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } } }
I20260810 14:43:33.895447 23789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:33.895900 23787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0af0fa8676a04a028a090223f5a181b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af0fa8676a04a028a090223f5a181b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44827 } } }
I20260810 14:43:33.896222 23787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:33.897747 23799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:33.904150 23799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:33.913686 23799 catalog_manager.cc:1347] Generated new cluster ID: becbd9a361894836ae3b79a554e979f5
I20260810 14:43:33.913815 23799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:33.927085 23799 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:33.927771 23799 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:33.937907 23799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0af0fa8676a04a028a090223f5a181b4: Generated new TSK 0
I20260810 14:43:33.939126 23799 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:33.951555 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23682
W20260810 14:43:33.953488 23768 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.52:57987
W20260810 14:43:33.954514 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23682: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:33.954768 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23682
2026-08-10T14:43:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2389 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:34 dist-test-slave-h6n0 krb5kdc[23819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:34 dist-test-slave-h6n0 krb5kdc[23819](info): set up 2 sockets
Aug 10 14:43:34 dist-test-slave-h6n0 krb5kdc[23819](info): commencing operation
krb5kdc: starting...
W20260810 14:43:35.375108 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.371s user 0.003s sys 0.055s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:35 dist-test-slave-h6n0 krb5kdc[23819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:35.750211 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43837
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:36.121270 23836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:36.121629 23836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:36.121806 23836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:36.129238 23836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:36.129479 23836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:36.129609 23836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:36.130087 23836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:36.130259 23836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:36.140889 23836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43837
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:36.142288 23836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:36.143671 23836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:36.156302 23844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:36.156749 23847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:36.156355 23851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:36.157547 23836 server_base.cc:1034] running on GCE node
I20260810 14:43:36.158950 23836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:36.160789 23836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:36.162128 23836 hybrid_clock.cc:648] HybridClock initialized: now 1786373016162027 us; error 65 us; skew 500 ppm
I20260810 14:43:36.162761 23836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:36 dist-test-slave-h6n0 krb5kdc[23819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373016, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:36.166625 23836 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:36.168982 23836 webserver.cc:460] Webserver started at http://127.0.0.1:33341/ using document root <none> and password file <none>
I20260810 14:43:36.169934 23836 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:36.170080 23836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:36.170670 23836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:36.173942 23836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "94996fef7f9f46ea9a4fb68f4854b32c"
format_stamp: "Formatted at 2026-08-10 14:43:36 on dist-test-slave-h6n0"
server_key: "52a58b476834df2e08dd14f6e3b17ac3"
server_key_iv: "ac4cf884ce7d9055b2553b4633704022"
server_key_version: "encryptionkey@0"
I20260810 14:43:36.175239 23836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "94996fef7f9f46ea9a4fb68f4854b32c"
format_stamp: "Formatted at 2026-08-10 14:43:36 on dist-test-slave-h6n0"
server_key: "52a58b476834df2e08dd14f6e3b17ac3"
server_key_iv: "ac4cf884ce7d9055b2553b4633704022"
server_key_version: "encryptionkey@0"
I20260810 14:43:36.181942 23836 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 14:43:36.185650 23861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:36.187353 23836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:43:36.187537 23836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "94996fef7f9f46ea9a4fb68f4854b32c"
format_stamp: "Formatted at 2026-08-10 14:43:36 on dist-test-slave-h6n0"
server_key: "52a58b476834df2e08dd14f6e3b17ac3"
server_key_iv: "ac4cf884ce7d9055b2553b4633704022"
server_key_version: "encryptionkey@0"
I20260810 14:43:36.187749 23836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:36.225947 23836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:36.232218 23836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:36.232570 23836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:36.257133 23836 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43837
I20260810 14:43:36.257193 23926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43837 every 8 connection(s)
I20260810 14:43:36.258728 23836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:36.262519 23928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:36.266436 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23836
I20260810 14:43:36.266669 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:36.267220 21725 external_mini_cluster.cc:1428] Setting key 788fa16d421ef50422f73edcc99b50e9
I20260810 14:43:36.280071 23928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c: Bootstrap starting.
I20260810 14:43:36.284152 23928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:36.285202 23928 log.cc:824] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:43:36 dist-test-slave-h6n0 krb5kdc[23819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 14:43:36.288208 23928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c: No bootstrap required, opened a new log
I20260810 14:43:36.296759 23935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:36.269157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:50633 (local address 127.0.0.1:43837)
0810 14:43:36.269934 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:36.269945 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:36.269987 (+ 42us) server_negotiation.cc:407] Connection header received
0810 14:43:36.272913 (+ 2926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:36.272944 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:36.273622 (+ 678us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:36.274125 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:36.276355 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:36.278531 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:36.280858 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:36.282502 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:36.282771 (+ 269us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:36.288669 (+ 5898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:36.288678 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:36.288692 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:36.288740 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:36.292291 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:36.293137 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:36.293144 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:36.293150 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:36.293319 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:36.293884 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:36.293888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:36.293890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:36.294472 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:36.294817 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:36.295363 (+ 546us) server_negotiation.cc:299] Negotiation successful
0810 14:43:36.295680 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":369,"threads_started":1}
I20260810 14:43:36.298647 23928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94996fef7f9f46ea9a4fb68f4854b32c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } }
I20260810 14:43:36.299182 23928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:36.299328 23928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94996fef7f9f46ea9a4fb68f4854b32c, State: Initialized, Role: FOLLOWER
I20260810 14:43:36.299942 23928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94996fef7f9f46ea9a4fb68f4854b32c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } }
I20260810 14:43:36.300263 23928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:36.300420 23928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:36.300603 23928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:36.301946 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:36.303211 23928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94996fef7f9f46ea9a4fb68f4854b32c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } }
I20260810 14:43:36.303699 23928 leader_election.cc:302] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94996fef7f9f46ea9a4fb68f4854b32c; no voters:
I20260810 14:43:36.304358 23928 leader_election.cc:288] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:36.304471 23937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:36.308669 23937 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [term 1 LEADER]: Becoming Leader. State: Replica: 94996fef7f9f46ea9a4fb68f4854b32c, State: Running, Role: LEADER
I20260810 14:43:36.309374 23937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94996fef7f9f46ea9a4fb68f4854b32c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } }
I20260810 14:43:36.311663 23939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94996fef7f9f46ea9a4fb68f4854b32c. Latest consensus state: current_term: 1 leader_uuid: "94996fef7f9f46ea9a4fb68f4854b32c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94996fef7f9f46ea9a4fb68f4854b32c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } } }
I20260810 14:43:36.314409 23939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:36.314488 23928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:36.314473 23943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94996fef7f9f46ea9a4fb68f4854b32c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94996fef7f9f46ea9a4fb68f4854b32c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43837 } } }
I20260810 14:43:36.314934 23943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:36.324988 23935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:36.307686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:38939 (local address 127.0.0.1:43837)
0810 14:43:36.307950 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:36.307958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:36.307988 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:43:36.308493 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:36.308498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:36.308571 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:36.308628 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:36.310489 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:36.312044 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:36.313799 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:36.314860 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:36.315153 (+ 293us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:36.318580 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:36.318589 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:36.318597 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:36.318647 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:36.321837 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:36.322667 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:36.322675 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:36.322679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:36.322856 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:36.323397 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:36.323402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:36.323404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:36.323745 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:36.324015 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:36.324427 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 14:43:36.324619 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260810 14:43:36.327062 23951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:36.332546 23951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:36.341351 23951 catalog_manager.cc:1347] Generated new cluster ID: 639d2f0685c14eff807bf71953fcb354
I20260810 14:43:36.341485 23951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:36.356226 23951 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:36.357452 23951 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:36.366604 23951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94996fef7f9f46ea9a4fb68f4854b32c: Generated new TSK 0
I20260810 14:43:36.367530 23951 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:36.382018 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23836
I20260810 14:43:36.383818 23921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:36.479700 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23836
W20260810 14:43:36.508219 23817 connection.cc:535] client connection to 127.0.0.1:43837 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43837): Connection reset by peer (error 104)
2026-08-10T14:43:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2529 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:36 dist-test-slave-h6n0 krb5kdc[23974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:36 dist-test-slave-h6n0 krb5kdc[23974](info): set up 2 sockets
Aug 10 14:43:36 dist-test-slave-h6n0 krb5kdc[23974](info): commencing operation
krb5kdc: starting...
W20260810 14:43:37.917806 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.385s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:38 dist-test-slave-h6n0 krb5kdc[23974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:38.259536 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44347
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:38.647184 23989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:38.647482 23989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:38.647653 23989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:38.654799 23989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:38.655169 23989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:38.655351 23989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:38.655545 23989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:38.655611 23989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:38.665779 23989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44347
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:38.667167 23989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:38.668504 23989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:38.681892 23998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:38.681860 23996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:38.681809 24001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:38.688251 23989 server_base.cc:1034] running on GCE node
I20260810 14:43:38.689831 23989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:38.692164 23989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:38.693524 23989 hybrid_clock.cc:648] HybridClock initialized: now 1786373018693474 us; error 33 us; skew 500 ppm
I20260810 14:43:38.693977 23989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:38 dist-test-slave-h6n0 krb5kdc[23974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:38.697854 23989 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:38.699700 23989 webserver.cc:460] Webserver started at http://127.0.0.1:46023/ using document root <none> and password file <none>
I20260810 14:43:38.700603 23989 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:38.700770 23989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:38.701107 23989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:38.704276 23989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "f8c804137d314a94991b947dc9f6c1a6"
format_stamp: "Formatted at 2026-08-10 14:43:38 on dist-test-slave-h6n0"
server_key: "7049c25be979c8038fc46b6909ff7dfb"
server_key_iv: "35a0a364e996e97faf2f372396c0aa63"
server_key_version: "encryptionkey@0"
I20260810 14:43:38.705072 23989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "f8c804137d314a94991b947dc9f6c1a6"
format_stamp: "Formatted at 2026-08-10 14:43:38 on dist-test-slave-h6n0"
server_key: "7049c25be979c8038fc46b6909ff7dfb"
server_key_iv: "35a0a364e996e97faf2f372396c0aa63"
server_key_version: "encryptionkey@0"
I20260810 14:43:38.710390 23989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:43:38.713835 24015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:38.715245 23989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:43:38.715446 23989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "f8c804137d314a94991b947dc9f6c1a6"
format_stamp: "Formatted at 2026-08-10 14:43:38 on dist-test-slave-h6n0"
server_key: "7049c25be979c8038fc46b6909ff7dfb"
server_key_iv: "35a0a364e996e97faf2f372396c0aa63"
server_key_version: "encryptionkey@0"
I20260810 14:43:38.715646 23989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:38.749118 23989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:38.755189 23989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:38.755537 23989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:38.778024 23989 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44347
I20260810 14:43:38.778074 24089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44347 every 8 connection(s)
I20260810 14:43:38.779578 23989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:38.782845 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 23989
I20260810 14:43:38.783070 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:38.783636 21725 external_mini_cluster.cc:1428] Setting key 5a63e871c353e229a5ee414323d557d1
I20260810 14:43:38.784819 24090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:38.800668 24095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:38.785774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:44991 (local address 127.0.0.1:44347)
0810 14:43:38.786694 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:38.786705 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:38.786754 (+ 49us) server_negotiation.cc:407] Connection header received
0810 14:43:38.788908 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:38.788938 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:38.789387 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:38.789748 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:38.790997 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:38.792797 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:38.794877 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:38.795613 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:38.795751 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:38.797357 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:38.797369 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:38.797393 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:38.798205 (+ 812us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:38.798331 (+ 126us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:38.798364 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:38.798636 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:38.799034 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 14:43:38.799391 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":390,"threads_started":1}
I20260810 14:43:38.800794 24090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6: Bootstrap starting.
I20260810 14:43:38.805824 24090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:38.806188 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:38.807766 24090 log.cc:824] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:38.811219 24090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6: No bootstrap required, opened a new log
I20260810 14:43:38.820760 24095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:38.812042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:55011 (local address 127.0.0.1:44347)
0810 14:43:38.812260 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:38.812266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:38.812627 (+ 361us) server_negotiation.cc:407] Connection header received
0810 14:43:38.812910 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:38.812915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:38.813021 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:38.813258 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:38.813973 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:38.815091 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:38.817209 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:38.817719 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:38.817835 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:38.819430 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:38.819434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:38.819438 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:38.819459 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:38.819499 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:38.819517 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:38.819608 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:38.820379 (+ 771us) server_negotiation.cc:299] Negotiation successful
0810 14:43:38.820442 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:43:38.824281 24090 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c804137d314a94991b947dc9f6c1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44347 } }
I20260810 14:43:38.825034 24090 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:38.825176 24090 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8c804137d314a94991b947dc9f6c1a6, State: Initialized, Role: FOLLOWER
I20260810 14:43:38.825886 24090 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c804137d314a94991b947dc9f6c1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44347 } }
I20260810 14:43:38.826148 24090 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:38.826248 24090 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:38.826383 24090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:38.828693 24090 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c804137d314a94991b947dc9f6c1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44347 } }
I20260810 14:43:38.829177 24090 leader_election.cc:302] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8c804137d314a94991b947dc9f6c1a6; no voters:
I20260810 14:43:38.829924 24090 leader_election.cc:288] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:38.830483 24102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:38.831717 24102 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [term 1 LEADER]: Becoming Leader. State: Replica: f8c804137d314a94991b947dc9f6c1a6, State: Running, Role: LEADER
I20260810 14:43:38.832293 24102 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c804137d314a94991b947dc9f6c1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44347 } }
I20260810 14:43:38.832432 24090 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:38.835500 24105 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8c804137d314a94991b947dc9f6c1a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c804137d314a94991b947dc9f6c1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44347 } } }
I20260810 14:43:38.835961 24105 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:38.835527 24108 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8c804137d314a94991b947dc9f6c1a6. Latest consensus state: current_term: 1 leader_uuid: "f8c804137d314a94991b947dc9f6c1a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c804137d314a94991b947dc9f6c1a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44347 } } }
I20260810 14:43:38.836382 24108 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:38.837146 24116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:38.842797 24116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:38.852046 24116 catalog_manager.cc:1347] Generated new cluster ID: f170527bb0024ad49a0f998bd991c802
I20260810 14:43:38.852172 24116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:38.864590 24116 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:38.865804 24116 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:38.885311 24116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8c804137d314a94991b947dc9f6c1a6: Generated new TSK 0
I20260810 14:43:38.886590 24116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:38.903395 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23989
W20260810 14:43:38.904772 24085 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.52:44991
W20260810 14:43:38.905728 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 23989: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:38.905912 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 23989
W20260810 14:43:38.932217 23970 connection.cc:535] client connection to 127.0.0.1:44347 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44347): Connection reset by peer (error 104)
2026-08-10T14:43:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:39 dist-test-slave-h6n0 krb5kdc[24136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:39 dist-test-slave-h6n0 krb5kdc[24136](info): set up 2 sockets
Aug 10 14:43:39 dist-test-slave-h6n0 krb5kdc[24136](info): commencing operation
krb5kdc: starting...
W20260810 14:43:40.319903 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:40 dist-test-slave-h6n0 krb5kdc[24136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:43:40.678803 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:41.055775 24155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:41.056094 24155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:41.056399 24155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:41.067147 24155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:41.067459 24155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:41.067533 24155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:41.067620 24155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:41.067682 24155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:41.081175 24155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41025
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:41.083443 24155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:41.085420 24155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:41.100447 24162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:41.100436 24163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:41.100744 24168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:41.101444 24155 server_base.cc:1034] running on GCE node
I20260810 14:43:41.102362 24155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:41.103930 24155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:41.105170 24155 hybrid_clock.cc:648] HybridClock initialized: now 1786373021105107 us; error 51 us; skew 500 ppm
I20260810 14:43:41.105572 24155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:41 dist-test-slave-h6n0 krb5kdc[24136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373021, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:41.110409 24155 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:43:41.112391 24155 webserver.cc:460] Webserver started at http://127.0.0.1:45939/ using document root <none> and password file <none>
I20260810 14:43:41.113476 24155 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:41.113647 24155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:41.114001 24155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:41.117163 24155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "1fd4fe7b99284023b779f35f13088a76"
format_stamp: "Formatted at 2026-08-10 14:43:41 on dist-test-slave-h6n0"
server_key: "dcdcbfd65ef0c9f0ebe9de2601515560"
server_key_iv: "39b14810cdd8465978a1f932b0b0105f"
server_key_version: "encryptionkey@0"
I20260810 14:43:41.117959 24155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "1fd4fe7b99284023b779f35f13088a76"
format_stamp: "Formatted at 2026-08-10 14:43:41 on dist-test-slave-h6n0"
server_key: "dcdcbfd65ef0c9f0ebe9de2601515560"
server_key_iv: "39b14810cdd8465978a1f932b0b0105f"
server_key_version: "encryptionkey@0"
I20260810 14:43:41.124017 24155 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 14:43:41.128659 24177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:41.130861 24155 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
I20260810 14:43:41.131099 24155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "1fd4fe7b99284023b779f35f13088a76"
format_stamp: "Formatted at 2026-08-10 14:43:41 on dist-test-slave-h6n0"
server_key: "dcdcbfd65ef0c9f0ebe9de2601515560"
server_key_iv: "39b14810cdd8465978a1f932b0b0105f"
server_key_version: "encryptionkey@0"
I20260810 14:43:41.131345 24155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:41.137941 24155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:41.143775 24155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:41.144186 24155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:41.169023 24155 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41025
I20260810 14:43:41.169116 24255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41025 every 8 connection(s)
I20260810 14:43:41.170678 24155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:41.172756 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 24155
I20260810 14:43:41.173010 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:41.173863 21725 external_mini_cluster.cc:1428] Setting key f6f695fc74dae3dac1c3f40c2b7b7f4a
I20260810 14:43:41.176541 24258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:41.182847 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:41.176243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:40837 (local address 127.0.0.1:41025)
0810 14:43:41.177179 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:41.177191 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:41.177235 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:43:41.179148 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:41.179170 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:41.179523 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:41.179894 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:41.181079 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:41.181086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:41.181102 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:41.181126 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:41.181203 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:41.181224 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:41.181413 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:41.181525 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 14:43:41.181735 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":447,"threads_started":1}
I20260810 14:43:41.188298 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:41.189512 24258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76: Bootstrap starting.
I20260810 14:43:41.192615 24258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:41.193789 24258 log.cc:824] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:41.196723 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:41.193625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:55573 (local address 127.0.0.1:41025)
0810 14:43:41.193828 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:41.193835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:41.194389 (+ 554us) server_negotiation.cc:407] Connection header received
0810 14:43:41.194629 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:41.194643 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:41.194701 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:41.194836 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:41.196015 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:41.196021 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:41.196026 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:41.196046 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:41.196098 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:41.196113 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:41.196250 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:41.196355 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 14:43:41.196423 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:43:41.197001 24258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76: No bootstrap required, opened a new log
I20260810 14:43:41.204864 24258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd4fe7b99284023b779f35f13088a76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260810 14:43:41.205295 24258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:41.205415 24258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd4fe7b99284023b779f35f13088a76, State: Initialized, Role: FOLLOWER
I20260810 14:43:41.205960 24258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd4fe7b99284023b779f35f13088a76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260810 14:43:41.206199 24258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:41.206349 24258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:41.206574 24258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:41.208714 24258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd4fe7b99284023b779f35f13088a76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260810 14:43:41.209208 24258 leader_election.cc:302] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fd4fe7b99284023b779f35f13088a76; no voters:
I20260810 14:43:41.209944 24258 leader_election.cc:288] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:41.210366 24273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:41.211337 24273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [term 1 LEADER]: Becoming Leader. State: Replica: 1fd4fe7b99284023b779f35f13088a76, State: Running, Role: LEADER
I20260810 14:43:41.211982 24273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd4fe7b99284023b779f35f13088a76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } }
I20260810 14:43:41.212309 24258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:41.215586 24275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fd4fe7b99284023b779f35f13088a76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd4fe7b99284023b779f35f13088a76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } } }
I20260810 14:43:41.216212 24275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:41.215619 24276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd4fe7b99284023b779f35f13088a76. Latest consensus state: current_term: 1 leader_uuid: "1fd4fe7b99284023b779f35f13088a76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd4fe7b99284023b779f35f13088a76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41025 } } }
I20260810 14:43:41.216773 24276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:41.219223 24285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:41.225242 24285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:41.235948 24285 catalog_manager.cc:1347] Generated new cluster ID: d8b954c3379e4562ba0e4708b5c04660
I20260810 14:43:41.236171 24285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:41.261539 24285 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:41.262180 24285 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:41.272682 24285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fd4fe7b99284023b779f35f13088a76: Generated new TSK 0
I20260810 14:43:41.273813 24285 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:41.280658 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24155
W20260810 14:43:41.282336 24248 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.52:40837
W20260810 14:43:41.283442 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24155: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:41.283609 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 24155
2026-08-10T14:43:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:41 dist-test-slave-h6n0 krb5kdc[24308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:41 dist-test-slave-h6n0 krb5kdc[24308](info): set up 2 sockets
Aug 10 14:43:41 dist-test-slave-h6n0 krb5kdc[24308](info): commencing operation
krb5kdc: starting...
W20260810 14:43:42.704599 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.371s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:42 dist-test-slave-h6n0 krb5kdc[24308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:43:43.072189 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:37165
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:46743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:37165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:43.454646 24329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:43.454953 24329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:43.455124 24329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:43.463438 24329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:43.463578 24329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:43.463639 24329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:43.463737 24329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:43.463814 24329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:43.476487 24329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:46743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:37165
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:37165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:43.478018 24329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:43.479679 24329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:43.492684 24339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:43.492815 24336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:43.493010 24337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:43.494089 24329 server_base.cc:1034] running on GCE node
I20260810 14:43:43.495167 24329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:43.497166 24329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:43.498410 24329 hybrid_clock.cc:648] HybridClock initialized: now 1786373023498342 us; error 58 us; skew 500 ppm
I20260810 14:43:43.498842 24329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:43 dist-test-slave-h6n0 krb5kdc[24308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373023, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:43.507612 24329 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:43:43.509441 24329 webserver.cc:460] Webserver started at http://127.21.55.126:41207/ using document root <none> and password file <none>
I20260810 14:43:43.510174 24329 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:43.510308 24329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:43.510686 24329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:43.513692 24329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "3b4391463ec746449c359414f52c1c4e"
format_stamp: "Formatted at 2026-08-10 14:43:43 on dist-test-slave-h6n0"
server_key: "9352ff08c9946c5187e37d5c56c6263f"
server_key_iv: "a1347e885a830bbecc543d6b9b854026"
server_key_version: "encryptionkey@0"
I20260810 14:43:43.514487 24329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "3b4391463ec746449c359414f52c1c4e"
format_stamp: "Formatted at 2026-08-10 14:43:43 on dist-test-slave-h6n0"
server_key: "9352ff08c9946c5187e37d5c56c6263f"
server_key_iv: "a1347e885a830bbecc543d6b9b854026"
server_key_version: "encryptionkey@0"
I20260810 14:43:43.520126 24329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:43:43.524302 24349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:43.525889 24329 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 14:43:43.526203 24329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "3b4391463ec746449c359414f52c1c4e"
format_stamp: "Formatted at 2026-08-10 14:43:43 on dist-test-slave-h6n0"
server_key: "9352ff08c9946c5187e37d5c56c6263f"
server_key_iv: "a1347e885a830bbecc543d6b9b854026"
server_key_version: "encryptionkey@0"
I20260810 14:43:43.526398 24329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:43.547950 24329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:43.554288 24329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:43.554647 24329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:43.576754 24329 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:37165
I20260810 14:43:43.576798 24427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:37165 every 8 connection(s)
I20260810 14:43:43.578209 24329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:43.582136 24428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:43.583734 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 24329
I20260810 14:43:43.583950 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:43.584537 21725 external_mini_cluster.cc:1428] Setting key b978d522e3be467badc957767cec0c15
I20260810 14:43:43.595940 24428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e: Bootstrap starting.
I20260810 14:43:43.598881 24428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e: Neither blocks nor log segments found. Creating new log.
Aug 10 14:43:43 dist-test-slave-h6n0 krb5kdc[24308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:43:43.599948 24428 log.cc:824] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:43.603055 24428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e: No bootstrap required, opened a new log
I20260810 14:43:43.611150 24428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4391463ec746449c359414f52c1c4e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37165 } }
I20260810 14:43:43.611650 24428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:43.611783 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:43.586959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34287 (local address 127.21.55.126:37165)
0810 14:43:43.587779 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:43.587790 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:43.587834 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:43:43.589504 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:43.589524 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:43.589913 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:43.590338 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:43.592096 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:43.593430 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:43.594756 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:43.595415 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:43.595639 (+ 224us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:43.601236 (+ 5597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:43.601247 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:43.601268 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:43.601329 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:43.605387 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:43.606692 (+ 1305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:43.606700 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:43.606708 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:43.606945 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:43.607485 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:43.607489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:43.607492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:43.608212 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:43.608660 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:43.609523 (+ 863us) server_negotiation.cc:299] Negotiation successful
0810 14:43:43.609975 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":358,"threads_started":1}
I20260810 14:43:43.611846 24428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b4391463ec746449c359414f52c1c4e, State: Initialized, Role: FOLLOWER
I20260810 14:43:43.612500 24428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4391463ec746449c359414f52c1c4e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37165 } }
I20260810 14:43:43.612802 24428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:43.612973 24428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:43.613184 24428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:43.615942 24428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4391463ec746449c359414f52c1c4e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37165 } }
I20260810 14:43:43.616528 24428 leader_election.cc:302] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b4391463ec746449c359414f52c1c4e; no voters:
I20260810 14:43:43.617214 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:43.617228 24428 leader_election.cc:288] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:43.617354 24434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:43.619426 24428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:43.624317 24434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [term 1 LEADER]: Becoming Leader. State: Replica: 3b4391463ec746449c359414f52c1c4e, State: Running, Role: LEADER
I20260810 14:43:43.625070 24434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4391463ec746449c359414f52c1c4e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37165 } }
I20260810 14:43:43.628355 24439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b4391463ec746449c359414f52c1c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4391463ec746449c359414f52c1c4e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37165 } } }
I20260810 14:43:43.628763 24439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:43.630208 24442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b4391463ec746449c359414f52c1c4e. Latest consensus state: current_term: 1 leader_uuid: "3b4391463ec746449c359414f52c1c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4391463ec746449c359414f52c1c4e" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37165 } } }
I20260810 14:43:43.630606 24442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e [sys.catalog]: This master's current role is: LEADER
W20260810 14:43:43.637146 24461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:43:43.637295 24461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:43:43.637444 24462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:43.639202 24462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:43.648447 24462 catalog_manager.cc:1347] Generated new cluster ID: f0dc8d805eeb42fe96d1a8bc8ef32340
I20260810 14:43:43.648571 24462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:43.652678 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:43.630919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52297 (local address 127.21.55.126:37165)
0810 14:43:43.631350 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:43.631359 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:43.635180 (+ 3821us) server_negotiation.cc:407] Connection header received
0810 14:43:43.635379 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:43.635384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:43.635460 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:43.635591 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:43.637637 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:43.638894 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:43.640802 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:43.641658 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:43.641845 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:43.645191 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:43.645198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:43.645204 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:43.645242 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:43.649284 (+ 4042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:43.650274 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:43.650280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:43.650284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:43.650456 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:43.650997 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:43.651003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:43.651006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:43.651401 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:43.651653 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:43.652086 (+ 433us) server_negotiation.cc:299] Negotiation successful
0810 14:43:43.652267 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260810 14:43:43.658982 24462 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:43.660681 24462 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:43.671087 24462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b4391463ec746449c359414f52c1c4e: Generated new TSK 0
I20260810 14:43:43.671911 24462 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:43.686618 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24329
I20260810 14:43:43.687839 24422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:43.784219 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 24329
W20260810 14:43:43.810227 24304 connection.cc:535] client connection to 127.21.55.126:37165 recv error: Network error: failed to read from TLS socket (remote: 127.21.55.126:37165): Connection reset by peer (error 104)
2026-08-10T14:43:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:43 dist-test-slave-h6n0 krb5kdc[24477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:43 dist-test-slave-h6n0 krb5kdc[24477](info): set up 2 sockets
Aug 10 14:43:43 dist-test-slave-h6n0 krb5kdc[24477](info): commencing operation
krb5kdc: starting...
W20260810 14:43:45.206296 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.377s user 0.002s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:45 dist-test-slave-h6n0 krb5kdc[24477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:43:45.526230 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38351
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:38351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:45.881672 24492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:45.881942 24492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:45.882117 24492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:45.889503 24492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:45.889616 24492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:45.889720 24492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:45.889809 24492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:45.889889 24492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:45.900626 24492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:38351
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:38351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:45.901940 24492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:45.903258 24492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:45.916653 24504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:45.916683 24506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:45.916735 24502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:45.917531 24492 server_base.cc:1034] running on GCE node
I20260810 14:43:45.918362 24492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:45.919792 24492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:45.921058 24492 hybrid_clock.cc:648] HybridClock initialized: now 1786373025921025 us; error 53 us; skew 500 ppm
I20260810 14:43:45.921423 24492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:45 dist-test-slave-h6n0 krb5kdc[24477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373025, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:45.924707 24492 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:43:45.926331 24492 webserver.cc:460] Webserver started at http://127.21.55.126:38811/ using document root <none> and password file <none>
I20260810 14:43:45.927057 24492 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:45.927162 24492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:45.927403 24492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:45.930338 24492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "f6b5baf1c8e0428c8176edfa2db54af4"
format_stamp: "Formatted at 2026-08-10 14:43:45 on dist-test-slave-h6n0"
server_key: "53344c5aedcbffbf7d90d2688f6735e4"
server_key_iv: "c26794943405c1b7946f58bd2d507ae0"
server_key_version: "encryptionkey@0"
I20260810 14:43:45.931092 24492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "f6b5baf1c8e0428c8176edfa2db54af4"
format_stamp: "Formatted at 2026-08-10 14:43:45 on dist-test-slave-h6n0"
server_key: "53344c5aedcbffbf7d90d2688f6735e4"
server_key_iv: "c26794943405c1b7946f58bd2d507ae0"
server_key_version: "encryptionkey@0"
I20260810 14:43:45.935890 24492 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 14:43:45.939339 24521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:45.940800 24492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:43:45.940949 24492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "f6b5baf1c8e0428c8176edfa2db54af4"
format_stamp: "Formatted at 2026-08-10 14:43:45 on dist-test-slave-h6n0"
server_key: "53344c5aedcbffbf7d90d2688f6735e4"
server_key_iv: "c26794943405c1b7946f58bd2d507ae0"
server_key_version: "encryptionkey@0"
I20260810 14:43:45.941138 24492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:45.963558 24492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:45.967635 24492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:45.967921 24492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:45.989081 24492 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:38351
I20260810 14:43:45.989399 24596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:38351 every 8 connection(s)
I20260810 14:43:45.990581 24492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:45.994431 24597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:45.999760 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 24492
I20260810 14:43:46.000141 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:46.000623 21725 external_mini_cluster.cc:1428] Setting key 791e6670c7e1d59557baf842a54d1fce
I20260810 14:43:46.005775 24597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4: Bootstrap starting.
I20260810 14:43:46.010071 24597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:46.011354 24597 log.cc:824] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:46.014856 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:46.002353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38139 (local address 127.21.55.126:38351)
0810 14:43:46.003252 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:46.003264 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:46.003310 (+ 46us) server_negotiation.cc:407] Connection header received
0810 14:43:46.004962 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:46.004991 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:46.005350 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:46.005639 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:46.006820 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:46.008299 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:46.010221 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:46.010954 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:46.011139 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:46.012476 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:46.012488 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:46.012512 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:46.013184 (+ 672us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:46.013260 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:46.013285 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:46.013412 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:46.013703 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 14:43:46.013883 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":437,"threads_started":1}
I20260810 14:43:46.015009 24597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4: No bootstrap required, opened a new log
I20260810 14:43:46.018561 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:46.024585 24597 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b5baf1c8e0428c8176edfa2db54af4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38351 } }
I20260810 14:43:46.025050 24597 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:46.025172 24597 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6b5baf1c8e0428c8176edfa2db54af4, State: Initialized, Role: FOLLOWER
I20260810 14:43:46.025805 24597 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b5baf1c8e0428c8176edfa2db54af4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38351 } }
I20260810 14:43:46.026042 24597 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:46.026202 24597 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:46.026402 24597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:46.029487 24597 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b5baf1c8e0428c8176edfa2db54af4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38351 } }
I20260810 14:43:46.030071 24597 leader_election.cc:302] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6b5baf1c8e0428c8176edfa2db54af4; no voters:
I20260810 14:43:46.030817 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:46.023681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58697 (local address 127.21.55.126:38351)
0810 14:43:46.023855 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:46.023863 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:46.024394 (+ 531us) server_negotiation.cc:407] Connection header received
0810 14:43:46.024572 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:46.024576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:46.024630 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:46.024703 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:46.025343 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:46.026175 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:46.028144 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:46.028674 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:46.028778 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:46.029948 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:46.029953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:46.029956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:46.029971 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:46.030030 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:46.030045 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:46.030128 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:46.030389 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 14:43:46.030446 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 14:43:46.030892 24597 leader_election.cc:288] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:46.031001 24610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:46.032302 24610 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [term 1 LEADER]: Becoming Leader. State: Replica: f6b5baf1c8e0428c8176edfa2db54af4, State: Running, Role: LEADER
I20260810 14:43:46.032943 24610 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b5baf1c8e0428c8176edfa2db54af4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38351 } }
I20260810 14:43:46.033311 24597 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:46.036134 24611 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6b5baf1c8e0428c8176edfa2db54af4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b5baf1c8e0428c8176edfa2db54af4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38351 } } }
I20260810 14:43:46.036227 24613 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6b5baf1c8e0428c8176edfa2db54af4. Latest consensus state: current_term: 1 leader_uuid: "f6b5baf1c8e0428c8176edfa2db54af4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6b5baf1c8e0428c8176edfa2db54af4" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 38351 } } }
I20260810 14:43:46.036638 24613 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:46.036525 24611 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:46.041157 24621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:46.047472 24621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:46.058678 24621 catalog_manager.cc:1347] Generated new cluster ID: 4364fdb112c54981a3ae4ca8b525276c
I20260810 14:43:46.058811 24621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:46.094800 24621 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:46.096104 24621 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:46.113545 24621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6b5baf1c8e0428c8176edfa2db54af4: Generated new TSK 0
I20260810 14:43:46.114825 24621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:46.121500 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24492
W20260810 14:43:46.123147 24592 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38139
W20260810 14:43:46.123903 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24492: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:46.124059 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 24492
W20260810 14:43:46.149442 24474 connection.cc:535] client connection to 127.21.55.126:38351 recv error: Network error: failed to read from TLS socket (remote: 127.21.55.126:38351): Connection reset by peer (error 104)
2026-08-10T14:43:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:46 dist-test-slave-h6n0 krb5kdc[24642](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:46 dist-test-slave-h6n0 krb5kdc[24642](info): set up 2 sockets
Aug 10 14:43:46 dist-test-slave-h6n0 krb5kdc[24642](info): commencing operation
krb5kdc: starting...
W20260810 14:43:47.523233 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.354s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:47 dist-test-slave-h6n0 krb5kdc[24642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:43:47.887714 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:36551
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:36551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:48.255502 24661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:48.255918 24661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:48.256022 24661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:48.264045 24661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:48.264173 24661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:48.264222 24661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:48.264307 24661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:48.264355 24661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:48.275290 24661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:36551
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:36551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:48.276714 24661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:48.278292 24661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:48.292399 24666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:48.292251 24669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:48.292315 24667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:48.293548 24661 server_base.cc:1034] running on GCE node
I20260810 14:43:48.294749 24661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:48.296492 24661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:48.297724 24661 hybrid_clock.cc:648] HybridClock initialized: now 1786373028297654 us; error 70 us; skew 500 ppm
I20260810 14:43:48.298112 24661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:48 dist-test-slave-h6n0 krb5kdc[24642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373028, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:48.301926 24661 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:43:48.303632 24661 webserver.cc:460] Webserver started at http://127.21.55.126:35233/ using document root <none> and password file <none>
I20260810 14:43:48.304333 24661 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:48.304469 24661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:48.304791 24661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:48.307852 24661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "40300e986ebb4341bf35f0de36bc5c2d"
format_stamp: "Formatted at 2026-08-10 14:43:48 on dist-test-slave-h6n0"
server_key: "6c4398db899abc4f6102273966ec2754"
server_key_iv: "4e754cb9ed11c215753515be1e21963a"
server_key_version: "encryptionkey@0"
I20260810 14:43:48.308647 24661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "40300e986ebb4341bf35f0de36bc5c2d"
format_stamp: "Formatted at 2026-08-10 14:43:48 on dist-test-slave-h6n0"
server_key: "6c4398db899abc4f6102273966ec2754"
server_key_iv: "4e754cb9ed11c215753515be1e21963a"
server_key_version: "encryptionkey@0"
I20260810 14:43:48.315237 24661 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 14:43:48.319327 24682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:48.321513 24661 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:43:48.321803 24661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "40300e986ebb4341bf35f0de36bc5c2d"
format_stamp: "Formatted at 2026-08-10 14:43:48 on dist-test-slave-h6n0"
server_key: "6c4398db899abc4f6102273966ec2754"
server_key_iv: "4e754cb9ed11c215753515be1e21963a"
server_key_version: "encryptionkey@0"
I20260810 14:43:48.322080 24661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:48.329561 24661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:48.334828 24661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:48.335170 24661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:48.358461 24661 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:36551
I20260810 14:43:48.358487 24758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:36551 every 8 connection(s)
I20260810 14:43:48.359962 24661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:48.365195 24761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:48.368841 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 24661
I20260810 14:43:48.369078 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:48.369678 21725 external_mini_cluster.cc:1428] Setting key 4669b2f1a3b096654b280d134cc60d7e
I20260810 14:43:48.379609 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:48.372435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45625 (local address 127.21.55.126:36551)
0810 14:43:48.373530 (+ 1095us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:48.373545 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:48.373591 (+ 46us) server_negotiation.cc:407] Connection header received
0810 14:43:48.375753 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:48.375778 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:48.376200 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:48.376654 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:48.377851 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:48.377858 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:48.377871 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:48.377887 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:48.377955 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:48.377979 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:48.378245 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:48.378383 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 14:43:48.378629 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":472,"threads_started":1}
I20260810 14:43:48.380385 24761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d: Bootstrap starting.
I20260810 14:43:48.383692 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:48.384708 24761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:48.386006 24761 log.cc:824] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:48.390489 24761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d: No bootstrap required, opened a new log
I20260810 14:43:48.392082 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:48.388843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40911 (local address 127.21.55.126:36551)
0810 14:43:48.389070 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:48.389079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:48.389509 (+ 430us) server_negotiation.cc:407] Connection header received
0810 14:43:48.389735 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:48.389740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:48.389805 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:48.389920 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:48.391284 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:48.391292 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:48.391298 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:48.391334 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:48.391386 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:48.391405 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:48.391548 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:48.391694 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 14:43:48.391765 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:43:48.400024 24761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40300e986ebb4341bf35f0de36bc5c2d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36551 } }
I20260810 14:43:48.400542 24761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:48.400660 24761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40300e986ebb4341bf35f0de36bc5c2d, State: Initialized, Role: FOLLOWER
I20260810 14:43:48.401371 24761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40300e986ebb4341bf35f0de36bc5c2d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36551 } }
I20260810 14:43:48.401643 24761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:48.401810 24761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:48.402005 24761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:48.405256 24761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40300e986ebb4341bf35f0de36bc5c2d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36551 } }
I20260810 14:43:48.405891 24761 leader_election.cc:302] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40300e986ebb4341bf35f0de36bc5c2d; no voters:
I20260810 14:43:48.406670 24761 leader_election.cc:288] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:48.406901 24777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:48.407685 24777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [term 1 LEADER]: Becoming Leader. State: Replica: 40300e986ebb4341bf35f0de36bc5c2d, State: Running, Role: LEADER
I20260810 14:43:48.408385 24777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40300e986ebb4341bf35f0de36bc5c2d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36551 } }
I20260810 14:43:48.409370 24761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:48.412397 24779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40300e986ebb4341bf35f0de36bc5c2d. Latest consensus state: current_term: 1 leader_uuid: "40300e986ebb4341bf35f0de36bc5c2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40300e986ebb4341bf35f0de36bc5c2d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36551 } } }
I20260810 14:43:48.413410 24779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:48.412514 24778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40300e986ebb4341bf35f0de36bc5c2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40300e986ebb4341bf35f0de36bc5c2d" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36551 } } }
I20260810 14:43:48.414680 24778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:48.424336 24784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:48.431623 24784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:48.443706 24784 catalog_manager.cc:1347] Generated new cluster ID: c16be2ca2eb44aeb92c93187b92a07a6
I20260810 14:43:48.443895 24784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:48.480551 24784 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:48.481206 24784 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:48.489295 24784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40300e986ebb4341bf35f0de36bc5c2d: Generated new TSK 0
I20260810 14:43:48.490759 24784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:48.504098 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24661
W20260810 14:43:48.505599 24752 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45625
W20260810 14:43:48.506378 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24661: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:48.506604 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 24661
2026-08-10T14:43:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2386 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:48 dist-test-slave-h6n0 krb5kdc[24801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:48 dist-test-slave-h6n0 krb5kdc[24801](info): set up 2 sockets
Aug 10 14:43:48 dist-test-slave-h6n0 krb5kdc[24801](info): commencing operation
krb5kdc: starting...
W20260810 14:43:49.912813 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.358s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:50 dist-test-slave-h6n0 krb5kdc[24801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:43:50.227437 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:44515
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:33195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:44515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:50.576992 24818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:50.577348 24818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:50.577513 24818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:50.584528 24818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:50.584681 24818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:50.584776 24818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:50.584929 24818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:50.585011 24818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:50.595268 24818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:33195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:44515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:44515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:50.596602 24818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:50.597942 24818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:50.610173 24828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:50.610982 24832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:50.611030 24830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:50.611902 24818 server_base.cc:1034] running on GCE node
I20260810 14:43:50.612717 24818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:50.614069 24818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:50.615334 24818 hybrid_clock.cc:648] HybridClock initialized: now 1786373030615288 us; error 51 us; skew 500 ppm
I20260810 14:43:50.615721 24818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:50 dist-test-slave-h6n0 krb5kdc[24801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373030, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:50.619077 24818 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:43:50.620649 24818 webserver.cc:460] Webserver started at http://127.21.55.126:33591/ using document root <none> and password file <none>
I20260810 14:43:50.621658 24818 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:50.621826 24818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:50.622140 24818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:50.625094 24818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "b73886ffd3c342929e52b2ceba2ad942"
format_stamp: "Formatted at 2026-08-10 14:43:50 on dist-test-slave-h6n0"
server_key: "806dc0b3392b223870830c21aedcead3"
server_key_iv: "189a31f03b6304b0f4da4b043c0ec90e"
server_key_version: "encryptionkey@0"
I20260810 14:43:50.625844 24818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "b73886ffd3c342929e52b2ceba2ad942"
format_stamp: "Formatted at 2026-08-10 14:43:50 on dist-test-slave-h6n0"
server_key: "806dc0b3392b223870830c21aedcead3"
server_key_iv: "189a31f03b6304b0f4da4b043c0ec90e"
server_key_version: "encryptionkey@0"
I20260810 14:43:50.631330 24818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260810 14:43:50.634974 24840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:50.636286 24818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:43:50.636474 24818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "b73886ffd3c342929e52b2ceba2ad942"
format_stamp: "Formatted at 2026-08-10 14:43:50 on dist-test-slave-h6n0"
server_key: "806dc0b3392b223870830c21aedcead3"
server_key_iv: "189a31f03b6304b0f4da4b043c0ec90e"
server_key_version: "encryptionkey@0"
I20260810 14:43:50.636677 24818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:50.654417 24818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:50.658697 24818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:50.658996 24818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:50.679570 24818 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:44515
I20260810 14:43:50.679603 24921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:44515 every 8 connection(s)
I20260810 14:43:50.680960 24818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:50.684795 24923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:50.689280 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 24818
I20260810 14:43:50.689484 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:50.689940 21725 external_mini_cluster.cc:1428] Setting key aa47ea99130108125aa9260b84f6c0f9
I20260810 14:43:50.697202 24923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942: Bootstrap starting.
I20260810 14:43:50.701062 24923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:50.702684 24923 log.cc:824] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:50.706094 24923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942: No bootstrap required, opened a new log
Aug 10 14:43:50 dist-test-slave-h6n0 krb5kdc[24801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:43:50.717000 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:50.691838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46435 (local address 127.21.55.126:44515)
0810 14:43:50.692712 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:50.692733 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:50.692784 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:43:50.695500 (+ 2716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:50.695527 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:50.696001 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:50.696420 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:50.697823 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:50.699884 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:50.701296 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:50.702377 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:50.702606 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:50.707999 (+ 5393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:50.708010 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:50.708028 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:50.708092 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:50.712415 (+ 4323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:50.713067 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:50.713076 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:50.713084 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:50.713288 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:50.713653 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:50.713658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:50.713662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:50.714406 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:50.714751 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:50.715215 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 14:43:50.715674 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":373,"threads_started":1}
I20260810 14:43:50.716862 24923 raft_consensus.cc:348] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73886ffd3c342929e52b2ceba2ad942" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44515 } }
I20260810 14:43:50.717343 24923 raft_consensus.cc:374] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:50.717450 24923 raft_consensus.cc:729] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b73886ffd3c342929e52b2ceba2ad942, State: Initialized, Role: FOLLOWER
I20260810 14:43:50.718118 24923 consensus_queue.cc:260] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73886ffd3c342929e52b2ceba2ad942" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44515 } }
I20260810 14:43:50.718416 24923 raft_consensus.cc:388] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:50.718591 24923 raft_consensus.cc:482] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:50.718792 24923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:50.721539 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:50.721556 24923 raft_consensus.cc:504] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73886ffd3c342929e52b2ceba2ad942" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44515 } }
I20260810 14:43:50.722160 24923 leader_election.cc:302] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b73886ffd3c342929e52b2ceba2ad942; no voters:
I20260810 14:43:50.722868 24923 leader_election.cc:288] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:50.723003 24928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:50.724761 24923 sys_catalog.cc:564] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:50.731765 24928 raft_consensus.cc:686] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [term 1 LEADER]: Becoming Leader. State: Replica: b73886ffd3c342929e52b2ceba2ad942, State: Running, Role: LEADER
I20260810 14:43:50.732482 24928 consensus_queue.cc:237] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73886ffd3c342929e52b2ceba2ad942" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44515 } }
I20260810 14:43:50.735826 24930 sys_catalog.cc:455] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b73886ffd3c342929e52b2ceba2ad942. Latest consensus state: current_term: 1 leader_uuid: "b73886ffd3c342929e52b2ceba2ad942" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73886ffd3c342929e52b2ceba2ad942" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44515 } } }
I20260810 14:43:50.735949 24929 sys_catalog.cc:455] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b73886ffd3c342929e52b2ceba2ad942" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73886ffd3c342929e52b2ceba2ad942" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 44515 } } }
I20260810 14:43:50.736287 24930 sys_catalog.cc:458] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:50.736340 24929 sys_catalog.cc:458] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942 [sys.catalog]: This master's current role is: LEADER
W20260810 14:43:50.752130 24955 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:43:50.752292 24955 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:43:50.752492 24958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:50.754771 24958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:50.755455 24926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:50.733730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50693 (local address 127.21.55.126:44515)
0810 14:43:50.733898 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:50.733904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:50.734770 (+ 866us) server_negotiation.cc:407] Connection header received
0810 14:43:50.734976 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:50.734982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:50.735080 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:50.735165 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:50.737703 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:50.739220 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:50.741960 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:50.742938 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:50.743362 (+ 424us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:50.748194 (+ 4832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:50.748200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:50.748205 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:50.748244 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:50.752604 (+ 4360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:50.753256 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:50.753263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:50.753266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:50.753411 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:50.753824 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:50.753830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:50.753833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:50.754215 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:50.754487 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:50.754887 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 14:43:50.755126 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:43:50.765554 24958 catalog_manager.cc:1347] Generated new cluster ID: 16aebcc1021d4fd09b7b1335b09b4d8d
I20260810 14:43:50.765709 24958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:50.790385 24958 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:50.791679 24958 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:50.802181 24958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b73886ffd3c342929e52b2ceba2ad942: Generated new TSK 0
I20260810 14:43:50.803526 24958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:50.811965 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24818
I20260810 14:43:50.813009 24915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:43:50.907886 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 24818
2026-08-10T14:43:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:51 dist-test-slave-h6n0 krb5kdc[24973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:51 dist-test-slave-h6n0 krb5kdc[24973](info): set up 2 sockets
Aug 10 14:43:51 dist-test-slave-h6n0 krb5kdc[24973](info): commencing operation
krb5kdc: starting...
W20260810 14:43:52.316148 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.363s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:52 dist-test-slave-h6n0 krb5kdc[24973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:43:52.662408 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:37899
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:45405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:37899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:53.009913 24993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:53.010215 24993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:53.010396 24993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:53.017577 24993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:53.017695 24993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:53.017750 24993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:53.017853 24993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:53.017935 24993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:53.028458 24993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:45405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:37899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:37899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:53.029712 24993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:53.031047 24993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:53.043301 25001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:53.043612 25002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:53.043833 25007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:53.044903 24993 server_base.cc:1034] running on GCE node
I20260810 14:43:53.045699 24993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:53.047185 24993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:53.048384 24993 hybrid_clock.cc:648] HybridClock initialized: now 1786373033048332 us; error 49 us; skew 500 ppm
I20260810 14:43:53.048732 24993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:53 dist-test-slave-h6n0 krb5kdc[24973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373033, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:53.051894 24993 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:43:53.053365 24993 webserver.cc:460] Webserver started at http://127.21.55.126:39785/ using document root <none> and password file <none>
I20260810 14:43:53.053999 24993 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:53.054105 24993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:53.054354 24993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:53.057327 24993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "ef1fb98580c946a78b61a2998e0c97ba"
format_stamp: "Formatted at 2026-08-10 14:43:53 on dist-test-slave-h6n0"
server_key: "ec86e8b944d8500726f45771d38319a5"
server_key_iv: "868ca1e21bcd903f7b64d9512eb490a1"
server_key_version: "encryptionkey@0"
I20260810 14:43:53.058087 24993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "ef1fb98580c946a78b61a2998e0c97ba"
format_stamp: "Formatted at 2026-08-10 14:43:53 on dist-test-slave-h6n0"
server_key: "ec86e8b944d8500726f45771d38319a5"
server_key_iv: "868ca1e21bcd903f7b64d9512eb490a1"
server_key_version: "encryptionkey@0"
I20260810 14:43:53.064600 24993 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 14:43:53.069650 25015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:53.071149 24993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:43:53.071352 24993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "ef1fb98580c946a78b61a2998e0c97ba"
format_stamp: "Formatted at 2026-08-10 14:43:53 on dist-test-slave-h6n0"
server_key: "ec86e8b944d8500726f45771d38319a5"
server_key_iv: "868ca1e21bcd903f7b64d9512eb490a1"
server_key_version: "encryptionkey@0"
I20260810 14:43:53.071563 24993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:53.084693 24993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:53.089915 24993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:53.090329 24993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:53.112310 24993 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:37899
I20260810 14:43:53.112314 25095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:37899 every 8 connection(s)
I20260810 14:43:53.113819 24993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:53.117810 25097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:53.119263 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 24993
I20260810 14:43:53.119544 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:53.120024 21725 external_mini_cluster.cc:1428] Setting key c6acc2936ef27a2d0cde7d5bf9a9338f
I20260810 14:43:53.130637 25097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba: Bootstrap starting.
I20260810 14:43:53.134140 25097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:53.135390 25100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:53.121865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47253 (local address 127.21.55.126:37899)
0810 14:43:53.122699 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:53.122713 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:53.122758 (+ 45us) server_negotiation.cc:407] Connection header received
0810 14:43:53.124341 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:53.124360 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:53.124748 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:53.125121 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:53.125645 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:53.127062 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:53.128962 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:53.129977 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:53.130294 (+ 317us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:53.131921 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:53.131931 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:53.131950 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:53.132725 (+ 775us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:53.132838 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:53.132871 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:53.133024 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:53.133314 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 14:43:53.133710 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":344,"threads_started":1}
I20260810 14:43:53.135782 25097 log.cc:824] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:53.139364 25097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba: No bootstrap required, opened a new log
I20260810 14:43:53.139966 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:53.148259 25097 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1fb98580c946a78b61a2998e0c97ba" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37899 } }
I20260810 14:43:53.148677 25097 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:53.148813 25097 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1fb98580c946a78b61a2998e0c97ba, State: Initialized, Role: FOLLOWER
I20260810 14:43:53.149443 25097 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1fb98580c946a78b61a2998e0c97ba" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37899 } }
I20260810 14:43:53.149703 25097 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:53.149878 25097 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:53.150077 25097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:53.151312 25100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:53.144623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51081 (local address 127.21.55.126:37899)
0810 14:43:53.144826 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:53.144836 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:53.145238 (+ 402us) server_negotiation.cc:407] Connection header received
0810 14:43:53.145563 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:53.145568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:53.145638 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:53.145770 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:53.146321 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:53.147136 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:53.148476 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:53.149081 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:53.149218 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:53.150477 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:53.150482 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:53.150486 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:53.150502 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:53.150598 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:53.150621 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:53.150738 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:53.150880 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 14:43:53.150998 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 14:43:53.152968 25097 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1fb98580c946a78b61a2998e0c97ba" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37899 } }
I20260810 14:43:53.153471 25097 leader_election.cc:302] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef1fb98580c946a78b61a2998e0c97ba; no voters:
I20260810 14:43:53.154750 25097 leader_election.cc:288] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:53.154790 25108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:53.156051 25108 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [term 1 LEADER]: Becoming Leader. State: Replica: ef1fb98580c946a78b61a2998e0c97ba, State: Running, Role: LEADER
I20260810 14:43:53.156718 25108 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1fb98580c946a78b61a2998e0c97ba" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37899 } }
I20260810 14:43:53.157475 25097 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:53.160408 25112 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef1fb98580c946a78b61a2998e0c97ba. Latest consensus state: current_term: 1 leader_uuid: "ef1fb98580c946a78b61a2998e0c97ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1fb98580c946a78b61a2998e0c97ba" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37899 } } }
I20260810 14:43:53.160301 25109 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef1fb98580c946a78b61a2998e0c97ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1fb98580c946a78b61a2998e0c97ba" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 37899 } } }
I20260810 14:43:53.160728 25109 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:53.160724 25112 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:53.163417 25118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:53.170383 25118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:53.181380 25118 catalog_manager.cc:1347] Generated new cluster ID: 22fe345af909478497d47d6cd881695b
I20260810 14:43:53.181517 25118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:53.195955 25118 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:53.197669 25118 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:53.209717 25118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef1fb98580c946a78b61a2998e0c97ba: Generated new TSK 0
I20260810 14:43:53.210597 25118 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:53.219450 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24993
W20260810 14:43:53.221135 25089 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47253
W20260810 14:43:53.222014 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 24993: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:53.222250 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 24993
2026-08-10T14:43:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:53 dist-test-slave-h6n0 krb5kdc[25147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:53 dist-test-slave-h6n0 krb5kdc[25147](info): set up 2 sockets
Aug 10 14:43:53 dist-test-slave-h6n0 krb5kdc[25147](info): commencing operation
krb5kdc: starting...
W20260810 14:43:54.617359 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:54 dist-test-slave-h6n0 krb5kdc[25147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:43:54.931435 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:34707
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:37751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:34707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:55.285326 25167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:55.285696 25167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:55.285884 25167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:55.293005 25167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:55.293169 25167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:55.293282 25167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:55.293395 25167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:55.293489 25167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:55.304284 25167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:37751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:34707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:34707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:55.305766 25167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:55.307129 25167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:55.320672 25174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:55.320588 25173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:55.320643 25179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:55.321638 25167 server_base.cc:1034] running on GCE node
I20260810 14:43:55.322468 25167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:55.324002 25167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:55.325244 25167 hybrid_clock.cc:648] HybridClock initialized: now 1786373035325201 us; error 52 us; skew 500 ppm
I20260810 14:43:55.325641 25167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:55 dist-test-slave-h6n0 krb5kdc[25147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373035, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:55.334499 25167 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:43:55.336323 25167 webserver.cc:460] Webserver started at http://127.21.55.126:39231/ using document root <none> and password file <none>
I20260810 14:43:55.337145 25167 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:55.337265 25167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:55.337524 25167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:55.340524 25167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "015952ac074248338a0415cd5f3ac032"
format_stamp: "Formatted at 2026-08-10 14:43:55 on dist-test-slave-h6n0"
server_key: "d5608ddce403a5f7dd35449d9dfd257b"
server_key_iv: "345bc9bf76662405f24e8ab57144995c"
server_key_version: "encryptionkey@0"
I20260810 14:43:55.341250 25167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "015952ac074248338a0415cd5f3ac032"
format_stamp: "Formatted at 2026-08-10 14:43:55 on dist-test-slave-h6n0"
server_key: "d5608ddce403a5f7dd35449d9dfd257b"
server_key_iv: "345bc9bf76662405f24e8ab57144995c"
server_key_version: "encryptionkey@0"
I20260810 14:43:55.347061 25167 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 14:43:55.351614 25188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:55.353994 25167 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260810 14:43:55.354286 25167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "015952ac074248338a0415cd5f3ac032"
format_stamp: "Formatted at 2026-08-10 14:43:55 on dist-test-slave-h6n0"
server_key: "d5608ddce403a5f7dd35449d9dfd257b"
server_key_iv: "345bc9bf76662405f24e8ab57144995c"
server_key_version: "encryptionkey@0"
I20260810 14:43:55.354596 25167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:55.361917 25167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:55.367185 25167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:55.367509 25167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:55.387490 25167 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:34707
I20260810 14:43:55.387539 25275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:34707 every 8 connection(s)
I20260810 14:43:55.388940 25167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:55.392616 25276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:55.394778 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 25167
I20260810 14:43:55.394982 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:55.395570 21725 external_mini_cluster.cc:1428] Setting key ff4aa7f6ce298fddf71f6eb7b7d70f51
I20260810 14:43:55.404585 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:55.398199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59093 (local address 127.21.55.126:34707)
0810 14:43:55.399251 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:55.399262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:55.399308 (+ 46us) server_negotiation.cc:407] Connection header received
0810 14:43:55.400967 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:55.400990 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:55.401319 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:55.401657 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:55.402913 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:55.402920 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:55.402943 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:55.402958 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:55.403050 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:55.403080 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:55.403280 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:55.403380 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 14:43:55.403630 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":490,"threads_started":1}
I20260810 14:43:55.405766 25276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032: Bootstrap starting.
I20260810 14:43:55.408447 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:55.409348 25276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:55.410631 25276 log.cc:824] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:55.416003 25276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032: No bootstrap required, opened a new log
I20260810 14:43:55.420076 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:55.416187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36847 (local address 127.21.55.126:34707)
0810 14:43:55.416473 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:55.416482 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:55.416977 (+ 495us) server_negotiation.cc:407] Connection header received
0810 14:43:55.417394 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:55.417399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:55.417459 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:55.417560 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:55.418996 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:55.419003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:55.419007 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:43:55.419032 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:55.419100 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:43:55.419141 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:55.419308 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:55.419511 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 14:43:55.419635 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260810 14:43:55.425393 25276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015952ac074248338a0415cd5f3ac032" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34707 } }
I20260810 14:43:55.425819 25276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:43:55.425954 25276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015952ac074248338a0415cd5f3ac032, State: Initialized, Role: FOLLOWER
I20260810 14:43:55.426694 25276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015952ac074248338a0415cd5f3ac032" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34707 } }
I20260810 14:43:55.426963 25276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:43:55.427063 25276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:43:55.427205 25276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:55.429368 25276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015952ac074248338a0415cd5f3ac032" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34707 } }
I20260810 14:43:55.429857 25276 leader_election.cc:302] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 015952ac074248338a0415cd5f3ac032; no voters:
I20260810 14:43:55.430413 25276 leader_election.cc:288] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:43:55.430615 25291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:55.431319 25291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [term 1 LEADER]: Becoming Leader. State: Replica: 015952ac074248338a0415cd5f3ac032, State: Running, Role: LEADER
I20260810 14:43:55.431895 25291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015952ac074248338a0415cd5f3ac032" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34707 } }
I20260810 14:43:55.432906 25276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:55.434875 25293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "015952ac074248338a0415cd5f3ac032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015952ac074248338a0415cd5f3ac032" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34707 } } }
I20260810 14:43:55.435468 25293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:55.435374 25294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 015952ac074248338a0415cd5f3ac032. Latest consensus state: current_term: 1 leader_uuid: "015952ac074248338a0415cd5f3ac032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "015952ac074248338a0415cd5f3ac032" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 34707 } } }
I20260810 14:43:55.435715 25294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032 [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:55.437562 25302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:55.442333 25302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:43:55.451833 25302 catalog_manager.cc:1347] Generated new cluster ID: 56977bab52654a0fb862f73c44659eb7
I20260810 14:43:55.452008 25302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:55.490389 25302 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:55.491089 25302 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:55.501372 25302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 015952ac074248338a0415cd5f3ac032: Generated new TSK 0
I20260810 14:43:55.502322 25302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:43:55.511544 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 25167
W20260810 14:43:55.512914 25267 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59093
W20260810 14:43:55.513638 21725 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 25167: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:43:55.513795 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 25167
2026-08-10T14:43:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2292 ms)
[----------] 18 tests from AuthTokenIssuingTest (43559 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:43:55 dist-test-slave-h6n0 krb5kdc[25327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:43:55 dist-test-slave-h6n0 krb5kdc[25327](info): set up 2 sockets
Aug 10 14:43:55 dist-test-slave-h6n0 krb5kdc[25327](info): commencing operation
krb5kdc: starting...
W20260810 14:43:56.912987 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.352s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:43:57 dist-test-slave-h6n0 krb5kdc[25327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:43:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:43:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:43:57.258100 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:35991
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:35991,127.21.55.125:33511,127.21.55.124:42143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:43:57.649411 25351 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:43:57.649777 25351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:57.649878 25351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:57.649987 25351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:57.657382 25351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:57.657470 25351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:57.657565 25351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:57.657657 25351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:57.657747 25351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:57.667704 25351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:35991,127.21.55.125:33511,127.21.55.124:42143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:35991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:57.669037 25351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:57.670336 25351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:57.682296 25356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:57.682335 25359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:57.682356 25357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:57.683584 25351 server_base.cc:1034] running on GCE node
I20260810 14:43:57.684444 25351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:57.685917 25351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:57.687165 25351 hybrid_clock.cc:648] HybridClock initialized: now 1786373037687093 us; error 49 us; skew 500 ppm
I20260810 14:43:57.687462 25351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:57 dist-test-slave-h6n0 krb5kdc[25327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373037, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:57.691077 25351 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:43:57.692695 25351 webserver.cc:460] Webserver started at http://127.21.55.126:39315/ using document root <none> and password file <none>
I20260810 14:43:57.693384 25351 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:57.693532 25351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:57.693861 25351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:57.697176 25351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "c0650e5eb26c4484b0bda2558f849c5b"
format_stamp: "Formatted at 2026-08-10 14:43:57 on dist-test-slave-h6n0"
server_key: "1963c2610561c1768554ad1c044b4811"
server_key_iv: "bce4bee5f6beb957419e6e8fe38e3e13"
server_key_version: "encryptionkey@0"
I20260810 14:43:57.697983 25351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "c0650e5eb26c4484b0bda2558f849c5b"
format_stamp: "Formatted at 2026-08-10 14:43:57 on dist-test-slave-h6n0"
server_key: "1963c2610561c1768554ad1c044b4811"
server_key_iv: "bce4bee5f6beb957419e6e8fe38e3e13"
server_key_version: "encryptionkey@0"
I20260810 14:43:57.702994 25351 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:43:57.706420 25370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:57.708134 25351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:43:57.708343 25351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "c0650e5eb26c4484b0bda2558f849c5b"
format_stamp: "Formatted at 2026-08-10 14:43:57 on dist-test-slave-h6n0"
server_key: "1963c2610561c1768554ad1c044b4811"
server_key_iv: "bce4bee5f6beb957419e6e8fe38e3e13"
server_key_version: "encryptionkey@0"
I20260810 14:43:57.708581 25351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:57.745857 25351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:57.751051 25351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:57.751559 25351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:57.774973 25351 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:35991
I20260810 14:43:57.775077 25451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:35991 every 8 connection(s)
I20260810 14:43:57.776407 25351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:43:57.778940 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 25351
I20260810 14:43:57.779177 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:43:57.780051 21725 external_mini_cluster.cc:1428] Setting key 3349e84b2f4beb5caf7e87362e61623b
I20260810 14:43:57.781001 25452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:57.792183 25452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:43:57 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373037, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.55.125@KRBTEST.COM; defaulting to no policy
I20260810 14:43:57.815994 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:57.796432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:35991 (local address 127.0.0.1:47606)
0810 14:43:57.797775 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:57.797819 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:57.799676 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:57.800530 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:57.800537 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:57.801253 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:57.801589 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:57.801604 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:57.803211 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:57.803220 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:57.804245 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:57.804256 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:57.805149 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:57.805155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:57.805325 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:57.806060 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:57.806081 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:57.808582 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:57.811808 (+ 3226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:57.811820 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:57.811831 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:57.812205 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:57.812601 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:57.812607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:57.812610 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:57.812767 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:57.813581 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:57.813589 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:57.813808 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 14:43:57.814052 (+ 244us) client_negotiation.cc:240] Negotiation successful
0810 14:43:57.814746 (+ 694us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"mutex_wait_us":238,"thread_start_us":786,"threads_started":1}
I20260810 14:43:57.816037 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:57.796422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47606 (local address 127.21.55.126:35991)
0810 14:43:57.797362 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:57.797379 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:57.797959 (+ 580us) server_negotiation.cc:407] Connection header received
0810 14:43:57.800054 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:57.800060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:57.800289 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:57.800494 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:57.801800 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:57.803052 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:57.804397 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:57.804919 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:57.805139 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:57.808796 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:57.808805 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:57.808815 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:57.808846 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:57.811685 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:57.812333 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:57.812348 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:57.812351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:57.812489 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:57.812873 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:57.812880 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:57.812883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:57.813439 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:57.813644 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:57.814175 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 14:43:57.814621 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":641,"thread_start_us":424,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.125@KRBTEST.COM" created.
I20260810 14:43:57.819362 25452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:43:57.821043 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:57.820448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41092)
0810 14:43:57.820761 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:57.820918 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260810 14:43:57.821408 25376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
I20260810 14:43:57.823220 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:57.822814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41100)
0810 14:43:57.823025 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:57.823092 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260810 14:43:57.823555 25452 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:33511: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab.
Entry for principal kudu/127.21.55.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab.
I20260810 14:43:57.859786 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.125:33511
--webserver_interface=127.21.55.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab
--principal=kudu/127.21.55.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:35991,127.21.55.125:33511,127.21.55.124:42143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:43:57.873031 25452 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } attempt: 1
I20260810 14:43:57.874473 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:57.873980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41114)
0810 14:43:57.874185 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:57.874309 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 14:43:57.875695 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:57.875253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41118)
0810 14:43:57.875424 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:57.875519 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:43:57.875921 25452 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:33511: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
I20260810 14:43:57.944198 25452 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } attempt: 2
I20260810 14:43:57.945842 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:57.945227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41124)
0810 14:43:57.945533 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:57.945665 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 14:43:57.947098 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:57.946649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41126)
0810 14:43:57.946798 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:57.946900 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 14:43:57.947305 25452 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:33511: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
I20260810 14:43:58.038837 25452 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } attempt: 3
I20260810 14:43:58.040359 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.039761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41132)
0810 14:43:58.040026 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.040164 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 14:43:58.041618 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.041132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41134)
0810 14:43:58.041317 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.041419 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 14:43:58.041893 25452 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:33511: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
I20260810 14:43:58.185187 25452 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } attempt: 4
I20260810 14:43:58.186697 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.185926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41138)
0810 14:43:58.186336 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.186469 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260810 14:43:58.188022 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.187605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41140)
0810 14:43:58.187767 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.187871 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 14:43:58.188344 25452 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:33511: Network error: Client connection negotiation failed: client connection to 127.21.55.125:33511: connect: Connection refused (error 111)
W20260810 14:43:58.231549 25461 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:43:58.231992 25461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:58.232132 25461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:58.232286 25461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:58.242146 25461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:58.242349 25461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:58.242493 25461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:58.242633 25461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:58.242784 25461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:58.255152 25461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:35991,127.21.55.125:33511,127.21.55.124:42143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab
--principal=kudu/127.21.55.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.125:33511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.55.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:58.257117 25461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:58.259020 25461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:58.275170 25468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:58.275786 25470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:58.276618 25461 server_base.cc:1034] running on GCE node
W20260810 14:43:58.284242 25467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:58.285390 25461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:58.287097 25461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:58.288374 25461 hybrid_clock.cc:648] HybridClock initialized: now 1786373038288280 us; error 50 us; skew 500 ppm
I20260810 14:43:58.288745 25461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:58 dist-test-slave-h6n0 krb5kdc[25327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373038, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:58.292213 25461 init.cc:377] Logged in from keytab as kudu/127.21.55.125@KRBTEST.COM (short username kudu)
I20260810 14:43:58.293972 25461 webserver.cc:460] Webserver started at http://127.21.55.125:40081/ using document root <none> and password file <none>
I20260810 14:43:58.294747 25461 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:58.294879 25461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:58.295224 25461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:58.299759 25461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/data/instance:
uuid: "1437e7989535418f8fdc3c2989c4051c"
format_stamp: "Formatted at 2026-08-10 14:43:58 on dist-test-slave-h6n0"
server_key: "05bdfd25fc28cbabf01050094e0797ed"
server_key_iv: "a254751a259fae3adb84c5fa44efdc58"
server_key_version: "encryptionkey@0"
I20260810 14:43:58.300921 25461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/wal/instance:
uuid: "1437e7989535418f8fdc3c2989c4051c"
format_stamp: "Formatted at 2026-08-10 14:43:58 on dist-test-slave-h6n0"
server_key: "05bdfd25fc28cbabf01050094e0797ed"
server_key_iv: "a254751a259fae3adb84c5fa44efdc58"
server_key_version: "encryptionkey@0"
I20260810 14:43:58.308876 25461 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.006s
I20260810 14:43:58.313773 25486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:58.315768 25461 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 14:43:58.316099 25461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/wal
uuid: "1437e7989535418f8fdc3c2989c4051c"
format_stamp: "Formatted at 2026-08-10 14:43:58 on dist-test-slave-h6n0"
server_key: "05bdfd25fc28cbabf01050094e0797ed"
server_key_iv: "a254751a259fae3adb84c5fa44efdc58"
server_key_version: "encryptionkey@0"
I20260810 14:43:58.316413 25461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:58.343140 25461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:58.347820 25461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:58.348137 25461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:58.371263 25461 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.125:33511
I20260810 14:43:58.371304 25568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.125:33511 every 8 connection(s)
I20260810 14:43:58.372915 25461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
I20260810 14:43:58.377301 25571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:58.378815 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 25461
I20260810 14:43:58.379030 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-1/wal/instance
I20260810 14:43:58.379604 21725 external_mini_cluster.cc:1428] Setting key 2f97d70fd602e181da3a7a23642dbdc7
I20260810 14:43:58.387180 25571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:43:58 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373038, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.55.124@KRBTEST.COM; defaulting to no policy
I20260810 14:43:58.418768 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.390627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47618 (local address 127.21.55.126:35991)
0810 14:43:58.391182 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:58.391189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:58.392842 (+ 1653us) server_negotiation.cc:407] Connection header received
0810 14:43:58.395354 (+ 2512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:58.395362 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:58.395461 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:58.395605 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:58.397215 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.398459 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.401269 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.402063 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.402221 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:58.408283 (+ 6062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:58.408294 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:58.408302 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:58.408367 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:58.413494 (+ 5127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:58.415351 (+ 1857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:58.415359 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:58.415366 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:58.415535 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:58.416216 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:58.416227 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:58.416232 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:58.416940 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:58.417188 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:58.418149 (+ 961us) server_negotiation.cc:299] Negotiation successful
0810 14:43:58.418418 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":261,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.124@KRBTEST.COM" created.
I20260810 14:43:58.420889 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.391277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:35991 (local address 127.0.0.1:47618)
0810 14:43:58.392629 (+ 1352us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.392682 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:58.395064 (+ 2382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:58.395817 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:58.395825 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:58.396407 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:58.396969 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:58.396987 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.398864 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.398871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:58.401015 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:58.401044 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.402391 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.402399 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:58.402737 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:58.403930 (+ 1193us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:58.403971 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:58.407929 (+ 3958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:58.413712 (+ 5783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:58.413721 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:58.413738 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:58.415124 (+ 1386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:58.415692 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:58.415696 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:58.415699 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:58.415915 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:58.417147 (+ 1232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:58.417154 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:58.417655 (+ 501us) client_negotiation.cc:769] Sending connection context
0810 14:43:58.417967 (+ 312us) client_negotiation.cc:240] Negotiation successful
0810 14:43:58.418579 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"thread_start_us":723,"threads_started":1}
I20260810 14:43:58.422999 25571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:43:58 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373038, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:43:58.445974 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.424013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41142)
0810 14:43:58.424259 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.424281 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:58.424394 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:58.425662 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:58.425667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:58.426182 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:58.426426 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:58.426438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.429012 (+ 2574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.429017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:58.429899 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:58.429911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.431244 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.431249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:58.431417 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:58.432554 (+ 1137us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:58.432591 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:58.436263 (+ 3672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:58.441560 (+ 5297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:58.441569 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:58.441576 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:58.442399 (+ 823us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:58.443291 (+ 892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:58.443300 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:58.443307 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:58.443549 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:58.444420 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:58.444428 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:58.444615 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 14:43:58.444788 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 14:43:58.445352 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":10}
I20260810 14:43:58.446010 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.424036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41142 (local address 127.21.55.125:33511)
0810 14:43:58.425023 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:58.425033 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:58.425078 (+ 45us) server_negotiation.cc:407] Connection header received
0810 14:43:58.425138 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:58.425148 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:58.425390 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:58.425552 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:58.426795 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.428808 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.430054 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.431064 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.431216 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:58.436627 (+ 5411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:58.436647 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:58.436671 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:58.436732 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:58.441336 (+ 4604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:58.442725 (+ 1389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:58.442734 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:58.442737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:58.442934 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:58.443746 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:58.443751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:58.443754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:58.444218 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:58.444437 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:58.445262 (+ 825us) server_negotiation.cc:299] Negotiation successful
0810 14:43:58.445566 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":658,"threads_started":1}
I20260810 14:43:58.450043 25571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:43:58.452138 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.451408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46628)
0810 14:43:58.451668 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.451815 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260810 14:43:58.452517 25489 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
I20260810 14:43:58.454706 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.454286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46638)
0810 14:43:58.454466 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.454599 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 14:43:58.455034 25571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:42143: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab.
Entry for principal kudu/127.21.55.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab.
I20260810 14:43:58.472215 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.124:42143
--webserver_interface=127.21.55.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab
--principal=kudu/127.21.55.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:44255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:35991,127.21.55.125:33511,127.21.55.124:42143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:43:58.487905 25452 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } attempt: 5
Aug 10 14:43:58 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373037, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:43:58.504407 25571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } attempt: 1
I20260810 14:43:58.506028 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.488944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:41156)
0810 14:43:58.489237 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.489265 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:58.489429 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:58.489977 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:58.489981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:58.490486 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:58.490906 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:58.490917 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.492464 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.492471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:58.493589 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:58.493605 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.494825 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.494831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:58.495017 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:58.495707 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:58.495724 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:58.498391 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:58.503116 (+ 4725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:58.503121 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:58.503124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:58.503566 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:58.504016 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:58.504020 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:58.504022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:58.504158 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:58.504916 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:58.504922 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:58.505128 (+ 206us) client_negotiation.cc:769] Sending connection context
0810 14:43:58.505302 (+ 174us) client_negotiation.cc:240] Negotiation successful
0810 14:43:58.505643 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 14:43:58.506059 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.505164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46648)
0810 14:43:58.505535 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.505744 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260810 14:43:58.506179 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.489187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41156 (local address 127.21.55.125:33511)
0810 14:43:58.489478 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:58.489488 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:58.489516 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:43:58.489593 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:58.489602 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:58.489719 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:58.489857 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:58.491081 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.492251 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.493767 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.494666 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.494809 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:58.498662 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:58.498672 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:58.498679 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:58.498725 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:58.502927 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:58.503700 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:58.503708 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:58.503713 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:58.503889 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:58.504276 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:58.504284 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:58.504314 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:58.504766 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:58.504949 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:58.505513 (+ 564us) server_negotiation.cc:299] Negotiation successful
0810 14:43:58.505753 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 14:43:58.507622 25452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:43:58.507763 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.507195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46654)
0810 14:43:58.507428 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.507549 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260810 14:43:58.508038 25571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:42143: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
I20260810 14:43:58.508834 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.508447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46658)
0810 14:43:58.508626 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.508709 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 14:43:58.509881 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.509567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46670)
0810 14:43:58.509719 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.509780 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 14:43:58.510107 25452 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:42143: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
I20260810 14:43:58.561342 25452 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } attempt: 1
I20260810 14:43:58.562777 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.562179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46672)
0810 14:43:58.562400 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.562492 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 14:43:58.563870 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.563463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46686)
0810 14:43:58.563612 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.563698 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 14:43:58.564087 25452 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:42143: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
I20260810 14:43:58.576327 25571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } attempt: 2
I20260810 14:43:58.577502 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.576986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46694)
0810 14:43:58.577203 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.577314 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 14:43:58.578718 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.578300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46696)
0810 14:43:58.578458 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.578578 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 14:43:58.578963 25571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:42143: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
I20260810 14:43:58.632309 25452 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } attempt: 2
I20260810 14:43:58.633539 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.633029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46708)
0810 14:43:58.633269 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.633368 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 14:43:58.634738 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.634233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46724)
0810 14:43:58.634402 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.634613 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 14:43:58.635068 25452 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:42143: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
I20260810 14:43:58.670508 25571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } attempt: 3
I20260810 14:43:58.671896 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.671221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46734)
0810 14:43:58.671523 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.671655 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 14:43:58.673278 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.672710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46744)
0810 14:43:58.672998 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.673104 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260810 14:43:58.673555 25571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:42143: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
I20260810 14:43:58.741307 25452 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } attempt: 3
I20260810 14:43:58.742736 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.742185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46748)
0810 14:43:58.742396 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.742501 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 14:43:58.743880 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.743443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46754)
0810 14:43:58.743686 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.743760 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260810 14:43:58.744100 25452 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:42143: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
I20260810 14:43:58.816816 25571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } attempt: 4
I20260810 14:43:58.818141 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.817541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46766)
0810 14:43:58.817782 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.817905 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 14:43:58.819221 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.818850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46768)
0810 14:43:58.819014 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.819098 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:43:58.819435 25571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:42143: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
W20260810 14:43:58.852736 25584 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:43:58.853067 25584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:43:58.853134 25584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:43:58.853217 25584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:43:58.860482 25584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:43:58.860577 25584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:43:58.860636 25584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:43:58.860682 25584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:43:58.860744 25584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:43:58.870745 25584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:44255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:35991,127.21.55.125:33511,127.21.55.124:42143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab
--principal=kudu/127.21.55.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.124:42143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.55.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:43:58.871991 25584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:43:58.873227 25584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:43:58.886627 25592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:58.886649 25593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:43:58.886885 25595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:43:58.887876 25584 server_base.cc:1034] running on GCE node
I20260810 14:43:58.888754 25584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:43:58.890158 25584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:43:58.891395 25584 hybrid_clock.cc:648] HybridClock initialized: now 1786373038891268 us; error 105 us; skew 500 ppm
I20260810 14:43:58.891728 25584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:43:58 dist-test-slave-h6n0 krb5kdc[25327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373038, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:43:58.895413 25584 init.cc:377] Logged in from keytab as kudu/127.21.55.124@KRBTEST.COM (short username kudu)
I20260810 14:43:58.896961 25584 webserver.cc:460] Webserver started at http://127.21.55.124:43905/ using document root <none> and password file <none>
I20260810 14:43:58.897650 25584 fs_manager.cc:359] Metadata directory not provided
I20260810 14:43:58.897740 25584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:43:58.898149 25584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:43:58.901191 25584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/data/instance:
uuid: "e7d585d1250a4eeea265e03c66f5d7b8"
format_stamp: "Formatted at 2026-08-10 14:43:58 on dist-test-slave-h6n0"
server_key: "14f2b7ecc143dec448cd70aa7d64ba63"
server_key_iv: "e2200ac92464ff93e12a619d16ac8556"
server_key_version: "encryptionkey@0"
I20260810 14:43:58.902004 25584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/wal/instance:
uuid: "e7d585d1250a4eeea265e03c66f5d7b8"
format_stamp: "Formatted at 2026-08-10 14:43:58 on dist-test-slave-h6n0"
server_key: "14f2b7ecc143dec448cd70aa7d64ba63"
server_key_iv: "e2200ac92464ff93e12a619d16ac8556"
server_key_version: "encryptionkey@0"
I20260810 14:43:58.908182 25584 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 14:43:58.913694 25603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:43:58.915767 25584 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260810 14:43:58.916016 25584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/wal
uuid: "e7d585d1250a4eeea265e03c66f5d7b8"
format_stamp: "Formatted at 2026-08-10 14:43:58 on dist-test-slave-h6n0"
server_key: "14f2b7ecc143dec448cd70aa7d64ba63"
server_key_iv: "e2200ac92464ff93e12a619d16ac8556"
server_key_version: "encryptionkey@0"
I20260810 14:43:58.916464 25584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:43:58.921330 25452 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } attempt: 4
I20260810 14:43:58.922446 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.922023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46772)
0810 14:43:58.922204 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.922310 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 14:43:58.923911 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.923148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:46784)
0810 14:43:58.923462 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.923644 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260810 14:43:58.924310 25452 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:42143: Network error: Client connection negotiation failed: client connection to 127.21.55.124:42143: connect: Connection refused (error 111)
I20260810 14:43:58.941077 25584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:43:58.947152 25584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:43:58.947608 25584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:43:58.970852 25584 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.124:42143
I20260810 14:43:58.971244 25690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.124:42143 every 8 connection(s)
I20260810 14:43:58.972863 25584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
I20260810 14:43:58.978228 25691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:43:58.982106 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 25584
I20260810 14:43:58.982311 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786372641103011-21725-0/minicluster-data/master-2/wal/instance
I20260810 14:43:58.982848 21725 external_mini_cluster.cc:1428] Setting key 3ed89dc6eb69f4ee62e75a80574e9049
I20260810 14:43:58.986071 25691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:43:58 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373038, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
Aug 10 14:43:58 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
I20260810 14:43:59.014658 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.984829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58768 (local address 127.21.55.124:42143)
0810 14:43:58.986762 (+ 1933us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:58.986777 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:58.986826 (+ 49us) server_negotiation.cc:407] Connection header received
0810 14:43:58.990610 (+ 3784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:58.990661 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:58.991125 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:58.991505 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:58.993416 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.995227 (+ 1811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.996935 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.997551 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.997691 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.004143 (+ 6452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:59.004151 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:59.004163 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:59.004199 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:59.007628 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.009008 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.009019 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.009022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.009213 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.010112 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.010119 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.010123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.011184 (+ 1061us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:59.011574 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:59.013258 (+ 1684us) server_negotiation.cc:299] Negotiation successful
0810 14:43:59.013649 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1697,"thread_start_us":1439,"threads_started":1}
I20260810 14:43:59.015621 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.989098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55570 (local address 127.21.55.126:35991)
0810 14:43:58.990635 (+ 1537us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:58.990640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:58.990658 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:43:58.991177 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:58.991190 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:58.991292 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:58.991493 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:58.993672 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.994855 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.997118 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.997647 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.997923 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.002985 (+ 5062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:59.002996 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:59.003004 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:59.003080 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:59.007396 (+ 4316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.008273 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.008280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.008284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.008516 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.009485 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.009493 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.009496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.009994 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:59.010224 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:59.014697 (+ 4473us) server_negotiation.cc:299] Negotiation successful
0810 14:43:59.015133 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1346,"thread_start_us":1158,"threads_started":1}
I20260810 14:43:59.015976 25697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:58.988954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:35991 (local address 127.0.0.1:55570)
0810 14:43:58.989791 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:58.989839 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:58.990610 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:58.991695 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:58.991704 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:58.992735 (+ 1031us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:58.993405 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:58.993428 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.995094 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.995104 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:58.996918 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:58.996934 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:58.997917 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:58.997926 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:58.998058 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:58.999004 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:58.999039 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:59.002613 (+ 3574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:59.007589 (+ 4976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:59.007601 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:59.007614 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:59.008154 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:59.008755 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:59.008762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:59.008766 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:59.009310 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:59.010338 (+ 1028us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:59.010351 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:59.011066 (+ 715us) client_negotiation.cc:769] Sending connection context
0810 14:43:59.011471 (+ 405us) client_negotiation.cc:240] Negotiation successful
0810 14:43:59.012259 (+ 788us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"spinlock_wait_cycles":58240,"thread_start_us":403,"threads_started":1}
I20260810 14:43:59.020757 25691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:43:59 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
Aug 10 14:43:59 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373038, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:43:59.038095 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.018682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55578 (local address 127.21.55.126:35991)
0810 14:43:59.018868 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.018874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.019118 (+ 244us) server_negotiation.cc:407] Connection header received
0810 14:43:59.019382 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.019392 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.019525 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.019658 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.021597 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.022867 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.024680 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.025224 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.025320 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.031039 (+ 5719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:59.031047 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:59.031051 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:59.031096 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:59.034210 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.035328 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.035335 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.035337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.035554 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.036120 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.036125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.036130 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.036547 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:59.036894 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:59.037592 (+ 698us) server_negotiation.cc:299] Negotiation successful
0810 14:43:59.037857 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 14:43:59.039700 25697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.021864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:33511 (local address 127.0.0.1:60256)
0810 14:43:59.023135 (+ 1271us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:59.023156 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:59.023265 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:59.023779 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:59.023784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:59.024260 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:59.024475 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:59.024488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.026017 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.026022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:59.027013 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:59.027025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.027709 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.027715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:59.027894 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.028517 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:59.028533 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:59.030941 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:59.036109 (+ 5168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:59.036114 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:59.036117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:59.036685 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:59.037460 (+ 775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:59.037465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:59.037485 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:59.037678 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:59.038659 (+ 981us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:59.038665 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:59.038835 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 14:43:59.039029 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 14:43:59.039327 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1137}
I20260810 14:43:59.039819 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.022312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.21.55.125:33511)
0810 14:43:59.022886 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.022892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.023214 (+ 322us) server_negotiation.cc:407] Connection header received
0810 14:43:59.023383 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.023389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.023476 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.023588 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.024733 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.025862 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.027152 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.027563 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.027695 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.031358 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:59.031366 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:59.031369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:59.031425 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:59.035920 (+ 4495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.036848 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.036855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.036858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.037233 (+ 375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.037951 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.037957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.037960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.038494 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:59.038702 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:59.039132 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 14:43:59.039352 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":308,"threads_started":1}
I20260810 14:43:59.041349 25691 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:43:59 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373038, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
Aug 10 14:43:59 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:43:59.056268 25697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.042063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:58780)
0810 14:43:59.042267 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:59.042287 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:59.042402 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:59.042920 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:59.042944 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:59.043495 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:59.043632 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:59.043641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.044603 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.044608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:59.046065 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:59.046094 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.046997 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.047003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:59.047227 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.047750 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:59.047768 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:59.050037 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:59.053581 (+ 3544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:59.053585 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:59.053587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:59.053969 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:59.054418 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:59.054421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:59.054424 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:59.054646 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:59.055216 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:59.055223 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:59.055404 (+ 181us) client_negotiation.cc:769] Sending connection context
0810 14:43:59.055548 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 14:43:59.055870 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":1}
I20260810 14:43:59.056389 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.042380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58780 (local address 127.21.55.124:42143)
0810 14:43:59.042581 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.042588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.042608 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:43:59.042659 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.042663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.042731 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.042901 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.043775 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.044466 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.046375 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.046852 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.046986 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.050214 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:59.050220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:59.050223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:59.050250 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:59.053422 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.054094 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.054101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.054107 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.054259 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.054763 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.054771 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.054775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.055068 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:59.055171 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:59.055706 (+ 535us) server_negotiation.cc:299] Negotiation successful
0810 14:43:59.056031 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":48}
I20260810 14:43:59.057776 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.040675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60260 (local address 127.21.55.125:33511)
0810 14:43:59.040836 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.040840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.040854 (+ 14us) server_negotiation.cc:407] Connection header received
0810 14:43:59.040927 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.040934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.041027 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.041128 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.042877 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.043728 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.045482 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.045914 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.046018 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.051054 (+ 5036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:59.051063 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:59.051069 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:59.051115 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:59.054673 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.055396 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.055400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.055402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.055554 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.056017 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.056022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.056025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.056360 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:59.056641 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:59.057214 (+ 573us) server_negotiation.cc:299] Negotiation successful
0810 14:43:59.057387 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 14:43:59.061501 25691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8: Bootstrap starting.
I20260810 14:43:59.062426 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:43:59.064576 25691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:59.065802 25691 log.cc:824] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:59.065089 25409 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c0650e5eb26c4484b0bda2558f849c5b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55578
I20260810 14:43:59.068691 25691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8: No bootstrap required, opened a new log
I20260810 14:43:59.075889 25691 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } }
I20260810 14:43:59.076380 25691 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d585d1250a4eeea265e03c66f5d7b8, State: Initialized, Role: FOLLOWER
I20260810 14:43:59.076874 25691 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } }
I20260810 14:43:59.077916 25704 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } } }
I20260810 14:43:59.078358 25704 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:43:59.078897 25691 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 14:43:59.088203 25718 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:43:59.088358 25718 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:43:59.086692 25584 master_runner.cc:186] Error getting master registration for 127.21.55.126:35991: OK, instance_id { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" instance_seqno: 1786373037769106 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:43:59.099453 25584 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:43:59.118713 25571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } attempt: 5
Aug 10 14:43:59 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373038, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
I20260810 14:43:59.137212 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.119592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58792 (local address 127.21.55.124:42143)
0810 14:43:59.119805 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.119810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.119868 (+ 58us) server_negotiation.cc:407] Connection header received
0810 14:43:59.120103 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.120107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.120236 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.120382 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.121557 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.122775 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.123925 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.124692 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.124816 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.129476 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:59.129482 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:59.129489 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:59.129531 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:59.133392 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.134760 (+ 1368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.134767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.134773 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.134940 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.135454 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.135459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.135463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.135973 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:59.136152 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:59.136579 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 14:43:59.136825 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:43:59.137220 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.119530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:58792)
0810 14:43:59.119777 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:59.119804 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:59.119969 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:59.120458 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:59.120463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:59.121104 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:59.121376 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:59.121392 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.122911 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.122917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:59.123784 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:59.123796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.124868 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.124886 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:59.125180 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.125985 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:59.126012 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:59.129253 (+ 3241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:59.133667 (+ 4414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:59.133676 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:59.133682 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:59.134622 (+ 940us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:59.135105 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:59.135113 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:59.135118 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:59.135332 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:59.136136 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:59.136140 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:59.136303 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 14:43:59.136473 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 14:43:59.136811 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 14:43:59.143314 25571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c: Bootstrap starting.
I20260810 14:43:59.146421 25571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:59.147495 25571 log.cc:824] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:59.150166 25571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c: No bootstrap required, opened a new log
I20260810 14:43:59.157312 25571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } }
I20260810 14:43:59.157662 25571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1437e7989535418f8fdc3c2989c4051c, State: Initialized, Role: FOLLOWER
I20260810 14:43:59.158149 25571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } }
I20260810 14:43:59.159204 25722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } } }
I20260810 14:43:59.159590 25722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:43:59.160367 25571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [sys.catalog]: configured and running, proceeding with master startup.
W20260810 14:43:59.170691 25740 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:43:59.170912 25740 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:43:59.168083 25461 master_runner.cc:186] Error getting master registration for 127.21.55.126:35991: OK, instance_id { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" instance_seqno: 1786373037769106 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:43:59.176450 25461 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 14:43:59.199998 25616 tablet.cc:2369] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:43:59.201542 25452 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } attempt: 5
Aug 10 14:43:59 dist-test-slave-h6n0 krb5kdc[25327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373037, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
I20260810 14:43:59.217235 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.202479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58798 (local address 127.21.55.124:42143)
0810 14:43:59.202650 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.202658 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.202737 (+ 79us) server_negotiation.cc:407] Connection header received
0810 14:43:59.202971 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.202977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.203052 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.203173 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.204131 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.204884 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.206707 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.207157 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.207303 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.211288 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:59.211299 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:59.211303 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:59.211336 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:59.214342 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.215203 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.215207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.215209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.215351 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.215796 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.215800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.215802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.216074 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:59.216234 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:59.216638 (+ 404us) server_negotiation.cc:299] Negotiation successful
0810 14:43:59.216854 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:43:59.217245 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.202311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:42143 (local address 127.0.0.1:58798)
0810 14:43:59.202642 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 14:43:59.202664 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:43:59.202799 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:43:59.203202 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:43:59.203207 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:43:59.203786 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:43:59.203995 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:59.204011 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.205028 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.205034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:59.206562 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:43:59.206586 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.207315 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.207320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:43:59.207554 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.208249 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:43:59.208271 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:43:59.211103 (+ 2832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:43:59.214550 (+ 3447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:59.214558 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:59.214565 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:59.215075 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:59.215474 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:43:59.215481 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:43:59.215485 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:43:59.215674 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:43:59.216200 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:43:59.216204 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:43:59.216373 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 14:43:59.216520 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 14:43:59.216798 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260810 14:43:59.223744 25452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b: Bootstrap starting.
I20260810 14:43:59.228009 25452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b: Neither blocks nor log segments found. Creating new log.
I20260810 14:43:59.229321 25452 log.cc:824] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:43:59.232697 25452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b: No bootstrap required, opened a new log
I20260810 14:43:59.240352 25452 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } }
I20260810 14:43:59.240747 25452 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0650e5eb26c4484b0bda2558f849c5b, State: Initialized, Role: FOLLOWER
I20260810 14:43:59.241701 25452 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } }
I20260810 14:43:59.248777 25743 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } } }
I20260810 14:43:59.249188 25743 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:43:59.250988 25452 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:43:59.251195 25409 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 14:43:59.251449 25409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [term 0 FOLLOWER]: Advancing to term 1
W20260810 14:43:59.255429 25378 tablet.cc:2369] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:43:59.255801 25409 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } }
I20260810 14:43:59.258809 25536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0650e5eb26c4484b0bda2558f849c5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1437e7989535418f8fdc3c2989c4051c"
I20260810 14:43:59.259465 25536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:59.265952 25536 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0650e5eb26c4484b0bda2558f849c5b in term 1.
I20260810 14:43:59.266928 25376 leader_election.cc:302] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1437e7989535418f8fdc3c2989c4051c, c0650e5eb26c4484b0bda2558f849c5b; no voters:
I20260810 14:43:59.267447 25743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:43:59.268918 25647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0650e5eb26c4484b0bda2558f849c5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e7d585d1250a4eeea265e03c66f5d7b8"
I20260810 14:43:59.269541 25647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:43:59.274165 25647 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0650e5eb26c4484b0bda2558f849c5b in term 1.
I20260810 14:43:59.275084 25409 leader_election.cc:288] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [CANDIDATE]: Term 1 election: Requested vote from peers 1437e7989535418f8fdc3c2989c4051c (127.21.55.125:33511), e7d585d1250a4eeea265e03c66f5d7b8 (127.21.55.124:42143)
I20260810 14:43:59.276326 25743 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [term 1 LEADER]: Becoming Leader. State: Replica: c0650e5eb26c4484b0bda2558f849c5b, State: Running, Role: LEADER
I20260810 14:43:59.277000 25743 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } }
I20260810 14:43:59.295456 25746 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0650e5eb26c4484b0bda2558f849c5b. Latest consensus state: current_term: 1 leader_uuid: "c0650e5eb26c4484b0bda2558f849c5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } } }
I20260810 14:43:59.296103 25746 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:59.307266 25754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:43:59.349042 25754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260810 14:43:59.353263 25499 tablet.cc:2369] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:43:59.366302 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.317385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55590 (local address 127.21.55.126:35991)
0810 14:43:59.317614 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.317622 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.332738 (+ 15116us) server_negotiation.cc:407] Connection header received
0810 14:43:59.333023 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.333029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.333135 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.333210 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.349371 (+ 16161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.350646 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.352634 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.353368 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.353498 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.357532 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:59.357539 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:59.357544 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:59.357588 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:59.362064 (+ 4476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.363071 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.363077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.363082 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.363230 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.363734 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.363739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.363743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.364177 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:59.364481 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:59.365566 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0810 14:43:59.365807 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 14:43:59.375813 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.327251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60264 (local address 127.21.55.125:33511)
0810 14:43:59.327480 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.327487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.348279 (+ 20792us) server_negotiation.cc:407] Connection header received
0810 14:43:59.348560 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.348566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.348661 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.348780 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.351282 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.352657 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.357839 (+ 5182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.358677 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.358802 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.367437 (+ 8635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:59.367451 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:59.367461 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:59.367555 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:59.371680 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.373006 (+ 1326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.373012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.373015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.373193 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.373678 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.373683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.373686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.374122 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:59.374471 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:59.375112 (+ 641us) server_negotiation.cc:299] Negotiation successful
0810 14:43:59.375390 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:43:59.371132 25772 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 14:43:59.384100 25647 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [term 1 FOLLOWER]: Refusing update from remote peer c0650e5eb26c4484b0bda2558f849c5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:43:59.384099 25536 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [term 1 FOLLOWER]: Refusing update from remote peer c0650e5eb26c4484b0bda2558f849c5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:43:59.385090 25743 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:43:59.385454 25746 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:43:59.399187 25695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:43:59.316114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58806 (local address 127.21.55.124:42143)
0810 14:43:59.316381 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.316391 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.330940 (+ 14549us) server_negotiation.cc:407] Connection header received
0810 14:43:59.331330 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.331340 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.331504 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.331617 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.367035 (+ 35418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.369627 (+ 2592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.375600 (+ 5973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:43:59.376329 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:43:59.376429 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:43:59.391807 (+ 15378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:43:59.391815 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:43:59.391819 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:43:59.391881 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:43:59.395691 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.396569 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.396575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.396579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.396721 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:43:59.397177 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:43:59.397187 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:43:59.397191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:43:59.397595 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:43:59.397864 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 14:43:59.398424 (+ 560us) server_negotiation.cc:299] Negotiation successful
0810 14:43:59.398726 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 14:43:59.415642 25704 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0650e5eb26c4484b0bda2558f849c5b. Latest consensus state: current_term: 1 leader_uuid: "c0650e5eb26c4484b0bda2558f849c5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } } }
I20260810 14:43:59.416005 25704 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:43:59.425520 25774 mvcc.cc:205] Tried to move back new op lower bound from 7316983969304039424 to 7316983968884416512. Current Snapshot: MvccSnapshot[applied={T|T < 7316983969304039424}]
I20260810 14:43:59.434358 25743 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0650e5eb26c4484b0bda2558f849c5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } } }
I20260810 14:43:59.434700 25743 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:59.435547 25754 catalog_manager.cc:1347] Generated new cluster ID: 50b7d1d5abb14fd49861bb0656212452
I20260810 14:43:59.435657 25754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:43:59.440337 25722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0650e5eb26c4484b0bda2558f849c5b. Latest consensus state: current_term: 1 leader_uuid: "c0650e5eb26c4484b0bda2558f849c5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } } }
I20260810 14:43:59.440774 25722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:43:59.443224 25746 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0650e5eb26c4484b0bda2558f849c5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } } }
I20260810 14:43:59.443869 25746 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b [sys.catalog]: This master's current role is: LEADER
I20260810 14:43:59.455168 25722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0650e5eb26c4484b0bda2558f849c5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } } }
I20260810 14:43:59.455451 25722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:43:59.459860 25704 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0650e5eb26c4484b0bda2558f849c5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0650e5eb26c4484b0bda2558f849c5b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 35991 } } peers { permanent_uuid: "1437e7989535418f8fdc3c2989c4051c" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 33511 } } peers { permanent_uuid: "e7d585d1250a4eeea265e03c66f5d7b8" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 42143 } } }
I20260810 14:43:59.460237 25704 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:43:59.485587 25754 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:43:59.487639 25754 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:43:59.506907 25754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0650e5eb26c4484b0bda2558f849c5b: Generated new TSK 0
I20260810 14:43:59.508167 25754 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:43:59.661216 25695 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:43:59.641194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58812 (local address 127.21.55.124:42143)
0810 14:43:59.641393 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.641400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.659099 (+ 17699us) server_negotiation.cc:407] Connection header received
0810 14:43:59.659531 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.659538 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.659624 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.659703 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.660941 (+ 1238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58812: BlockingRecv error: recv got EOF from 127.0.0.1:58812 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 14:43:59.663934 25700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:43:59.654833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60280 (local address 127.21.55.125:33511)
0810 14:43:59.655386 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.655396 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.662256 (+ 6860us) server_negotiation.cc:407] Connection header received
0810 14:43:59.662486 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.662493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.662607 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.662685 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.663661 (+ 976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60280: BlockingRecv error: recv got EOF from 127.0.0.1:60280 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 14:43:59.666422 25698 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:43:59.639161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55602 (local address 127.21.55.126:35991)
0810 14:43:59.639442 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.639450 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.657644 (+ 18194us) server_negotiation.cc:407] Connection header received
0810 14:43:59.658100 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.658108 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.658201 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.658308 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.666215 (+ 7907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55602: BlockingRecv error: recv got EOF from 127.0.0.1:55602 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 14:43:59.706015 25700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:43:59.696116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60284 (local address 127.21.55.125:33511)
0810 14:43:59.696343 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.696350 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.703787 (+ 7437us) server_negotiation.cc:407] Connection header received
0810 14:43:59.704343 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.704350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.704502 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.704584 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.705748 (+ 1164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60284: BlockingRecv error: recv got EOF from 127.0.0.1:60284 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 14:43:59.713682 25700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:43:59.711376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60298 (local address 127.21.55.125:33511)
0810 14:43:59.711564 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.711570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.712054 (+ 484us) server_negotiation.cc:407] Connection header received
0810 14:43:59.712233 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.712239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.712333 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.712464 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.713427 (+ 963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60298: BlockingRecv error: recv got EOF from 127.0.0.1:60298 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 14:43:59.722899 25700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:43:59.720457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60306 (local address 127.21.55.125:33511)
0810 14:43:59.720644 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.720649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.721181 (+ 532us) server_negotiation.cc:407] Connection header received
0810 14:43:59.721436 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.721442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.721539 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.721665 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.722640 (+ 975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60306: BlockingRecv error: recv got EOF from 127.0.0.1:60306 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260810 14:43:59.735561 25700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:43:59.733146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.21.55.125:33511)
0810 14:43:59.733372 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.733378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.733901 (+ 523us) server_negotiation.cc:407] Connection header received
0810 14:43:59.734101 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.734110 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.734216 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.734332 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.735273 (+ 941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60314: BlockingRecv error: recv got EOF from 127.0.0.1:60314 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260810 14:43:59.749190 25700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:43:59.746799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60324 (local address 127.21.55.125:33511)
0810 14:43:59.747010 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.747018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.747574 (+ 556us) server_negotiation.cc:407] Connection header received
0810 14:43:59.747766 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.747772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.747879 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.747996 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.748902 (+ 906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60324: BlockingRecv error: recv got EOF from 127.0.0.1:60324 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 14:43:59.771535 25700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:43:59.769152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60328 (local address 127.21.55.125:33511)
0810 14:43:59.769338 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.769345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.769953 (+ 608us) server_negotiation.cc:407] Connection header received
0810 14:43:59.770154 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.770161 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.770260 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.770378 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.771276 (+ 898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60328: BlockingRecv error: recv got EOF from 127.0.0.1:60328 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 14:43:59.809415 25700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:43:59.807302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60344 (local address 127.21.55.125:33511)
0810 14:43:59.807514 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.807523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.807913 (+ 390us) server_negotiation.cc:407] Connection header received
0810 14:43:59.808173 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.808179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.808294 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.808406 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.809170 (+ 764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60344: BlockingRecv error: recv got EOF from 127.0.0.1:60344 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 14:43:59.879179 25700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:43:59.877107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.21.55.125:33511)
0810 14:43:59.877314 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 14:43:59.877333 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 14:43:59.877658 (+ 325us) server_negotiation.cc:407] Connection header received
0810 14:43:59.877845 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:43:59.877852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:43:59.877957 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:43:59.878090 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:43:59.878934 (+ 844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60358: BlockingRecv error: recv got EOF from 127.0.0.1:60358 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260810 14:44:00.012997 25700 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:00.010700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60362 (local address 127.21.55.125:33511)
0810 14:44:00.010872 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:00.010877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:00.011279 (+ 402us) server_negotiation.cc:407] Connection header received
0810 14:44:00.011439 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:00.011444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:00.011516 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:00.011612 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:00.012634 (+ 1022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60362: BlockingRecv error: recv got EOF from 127.0.0.1:60362 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:44:00.090365 25718 catalog_manager.cc:1259] Loaded cluster ID: 50b7d1d5abb14fd49861bb0656212452
I20260810 14:44:00.090482 25718 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8: loading cluster ID for follower catalog manager: success
I20260810 14:44:00.093183 25718 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8: acquiring CA information for follower catalog manager: success
I20260810 14:44:00.095220 25718 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e7d585d1250a4eeea265e03c66f5d7b8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:44:00.172983 25740 catalog_manager.cc:1259] Loaded cluster ID: 50b7d1d5abb14fd49861bb0656212452
I20260810 14:44:00.173096 25740 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c: loading cluster ID for follower catalog manager: success
I20260810 14:44:00.175595 25740 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c: acquiring CA information for follower catalog manager: success
I20260810 14:44:00.177438 25740 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1437e7989535418f8fdc3c2989c4051c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:44:00.282833 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:00.272946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60378 (local address 127.21.55.125:33511)
0810 14:44:00.273287 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:00.273297 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:00.273745 (+ 448us) server_negotiation.cc:407] Connection header received
0810 14:44:00.274037 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:00.274046 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:00.274463 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:00.274699 (+ 236us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:00.275809 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:00.276745 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:00.279072 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:00.279951 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:00.280118 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:00.280923 (+ 805us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:00.281999 (+ 1076us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:00.282186 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:00.282329 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 14:44:00.282468 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 14:44:00.298974 25874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:00.291391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58824 (local address 127.21.55.124:42143)
0810 14:44:00.292059 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:00.292066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:00.292099 (+ 33us) server_negotiation.cc:407] Connection header received
0810 14:44:00.292443 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:00.292448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:00.292650 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:00.292895 (+ 245us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:00.293375 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:00.294240 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:00.296224 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:00.296780 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:00.296961 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:00.297545 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:00.298169 (+ 624us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:00.298358 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:00.298454 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 14:44:00.298597 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":336,"threads_started":1}
I20260810 14:44:00.321329 25700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:00.311412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.21.55.125:33511)
0810 14:44:00.311616 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:00.311625 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:00.313333 (+ 1708us) server_negotiation.cc:407] Connection header received
0810 14:44:00.313719 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:00.313728 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:00.313841 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:00.313943 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:00.315044 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:00.316387 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:00.318931 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:00.319502 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:00.319586 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:00.320388 (+ 802us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:00.320535 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:00.320623 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:00.320856 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 14:44:00.320972 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 14:44:00.321329 25874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:00.311797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.21.55.124:42143)
0810 14:44:00.312068 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:00.312076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:00.313252 (+ 1176us) server_negotiation.cc:407] Connection header received
0810 14:44:00.313779 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:00.313795 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:00.313915 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:00.313997 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:00.315208 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:00.316154 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:00.319141 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:00.319807 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:00.319942 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:00.320455 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:00.320616 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:00.320729 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:00.320912 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 14:44:00.320994 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:44:00.323064 25881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:00.311572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55610 (local address 127.21.55.126:35991)
0810 14:44:00.312103 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:00.312108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:00.314097 (+ 1989us) server_negotiation.cc:407] Connection header received
0810 14:44:00.315747 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:00.315754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:00.316034 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:00.316269 (+ 235us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:00.317042 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:00.318008 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:00.320239 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:00.320886 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:00.321004 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:00.321798 (+ 794us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:00.322320 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:00.322444 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:00.322655 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 14:44:00.322766 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":281,"threads_started":1}
I20260810 14:44:00.330153 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 25351
I20260810 14:44:00.331805 25442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:00.451277 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 25351
I20260810 14:44:00.484365 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 25461
I20260810 14:44:00.485617 25563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:00.601639 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 25461
I20260810 14:44:00.629171 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 25584
I20260810 14:44:00.630856 25679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:00.743490 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 25584
2026-08-10T14:44:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5238 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:44:00 dist-test-slave-h6n0 krb5kdc[25895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:44:00 dist-test-slave-h6n0 krb5kdc[25895](info): set up 2 sockets
Aug 10 14:44:00 dist-test-slave-h6n0 krb5kdc[25895](info): commencing operation
krb5kdc: starting...
W20260810 14:44:02.225189 21725 mini_kdc.cc:121] Time spent starting KDC: real 1.426s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/test-user.keytab.
Aug 10 14:44:02 dist-test-slave-h6n0 krb5kdc[25895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.55.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
Entry for principal kudu/127.21.55.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab.
I20260810 14:44:02.564962 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:36177
--webserver_interface=127.21.55.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:36177,127.21.55.125:34575,127.21.55.124:38775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:02.917544 25921 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:44:02.917870 25921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:02.917937 25921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:02.918021 25921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:02.925026 25921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:02.925114 25921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:02.925173 25921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:02.925216 25921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:02.925282 25921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:02.934979 25921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:36177,127.21.55.125:34575,127.21.55.124:38775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.126.keytab
--principal=kudu/127.21.55.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.126:36177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.55.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:02.936239 25921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:02.937500 25921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:02.950596 25932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:02.950519 25930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:02.950748 25928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:02.951560 25921 server_base.cc:1034] running on GCE node
I20260810 14:44:02.952389 25921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:02.953982 25921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:02.955261 25921 hybrid_clock.cc:648] HybridClock initialized: now 1786373042955177 us; error 45 us; skew 500 ppm
I20260810 14:44:02.955566 25921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:02 dist-test-slave-h6n0 krb5kdc[25895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373042, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:02.959178 25921 init.cc:377] Logged in from keytab as kudu/127.21.55.126@KRBTEST.COM (short username kudu)
I20260810 14:44:02.960629 25921 webserver.cc:460] Webserver started at http://127.21.55.126:45637/ using document root <none> and password file <none>
I20260810 14:44:02.961326 25921 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:02.961441 25921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:02.961774 25921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:02.964741 25921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/data/instance:
uuid: "b806ab558629443e818d5b33ba40d13b"
format_stamp: "Formatted at 2026-08-10 14:44:02 on dist-test-slave-h6n0"
server_key: "fe07ba342efd3954cb6e45f1b3e7f62e"
server_key_iv: "487c5303a4b67e3527b024c79b096743"
server_key_version: "encryptionkey@0"
I20260810 14:44:02.965556 25921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/wal/instance:
uuid: "b806ab558629443e818d5b33ba40d13b"
format_stamp: "Formatted at 2026-08-10 14:44:02 on dist-test-slave-h6n0"
server_key: "fe07ba342efd3954cb6e45f1b3e7f62e"
server_key_iv: "487c5303a4b67e3527b024c79b096743"
server_key_version: "encryptionkey@0"
I20260810 14:44:02.971365 25921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:44:02.975059 25942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:02.976470 25921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:44:02.976692 25921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/wal
uuid: "b806ab558629443e818d5b33ba40d13b"
format_stamp: "Formatted at 2026-08-10 14:44:02 on dist-test-slave-h6n0"
server_key: "fe07ba342efd3954cb6e45f1b3e7f62e"
server_key_iv: "487c5303a4b67e3527b024c79b096743"
server_key_version: "encryptionkey@0"
I20260810 14:44:02.976922 25921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:03.005842 25921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:03.010558 25921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:03.010893 25921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:03.034175 25921 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.126:36177
I20260810 14:44:03.034205 26024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.126:36177 every 8 connection(s)
I20260810 14:44:03.035712 25921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:03.038981 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 25921
I20260810 14:44:03.039191 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-0/wal/instance
I20260810 14:44:03.039726 26027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:03.039783 21725 external_mini_cluster.cc:1428] Setting key d42d901e04d7137ee1446fdb99cddc04
I20260810 14:44:03.049592 26027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:44:03 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373042, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.55.125@KRBTEST.COM; defaulting to no policy
I20260810 14:44:03.076628 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.053466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:36177 (local address 127.0.0.1:55590)
0810 14:44:03.054417 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.054477 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:03.056715 (+ 2238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:03.057637 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:03.057642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:03.058354 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:03.058771 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:03.058790 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.061220 (+ 2430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.061224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:03.062923 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:03.062936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.064286 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.064292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:03.064457 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:03.065243 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:03.065270 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:03.067699 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:03.072397 (+ 4698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:03.072408 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:03.072416 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:03.072812 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:03.073322 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:03.073325 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:03.073327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:03.073479 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:03.074453 (+ 974us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:03.074458 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:03.074931 (+ 473us) client_negotiation.cc:769] Sending connection context
0810 14:44:03.075198 (+ 267us) client_negotiation.cc:240] Negotiation successful
0810 14:44:03.075571 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":2,"thread_start_us":394,"threads_started":1}
I20260810 14:44:03.076646 26034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.053466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55590 (local address 127.21.55.126:36177)
0810 14:44:03.055386 (+ 1920us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:03.055392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:03.055440 (+ 48us) server_negotiation.cc:407] Connection header received
0810 14:44:03.057087 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:03.057097 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:03.057380 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:03.057607 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:03.058989 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.061046 (+ 2057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.063070 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.064048 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.064241 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:03.067952 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:03.067964 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:03.067981 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:03.068029 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:03.072226 (+ 4197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:03.072977 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:03.072988 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:03.072991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:03.073199 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:03.073605 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:03.073613 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:03.073616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:03.074293 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:03.074793 (+ 500us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:03.075361 (+ 568us) server_negotiation.cc:299] Negotiation successful
0810 14:44:03.075676 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":165,"server-negotiator.queue_time_us":1739,"spinlock_wait_cycles":16128,"thread_start_us":1416,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.125@KRBTEST.COM" created.
I20260810 14:44:03.079560 26027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:44:03.080791 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.080279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40714)
0810 14:44:03.080492 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.080630 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 14:44:03.081188 25946 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
I20260810 14:44:03.083099 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.082402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40728)
0810 14:44:03.082828 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.082923 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
W20260810 14:44:03.083375 26027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:34575: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab.
Entry for principal kudu/127.21.55.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab.
I20260810 14:44:03.115128 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.125:34575
--webserver_interface=127.21.55.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab
--principal=kudu/127.21.55.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:36177,127.21.55.125:34575,127.21.55.124:38775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:44:03.132798 26027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } attempt: 1
I20260810 14:44:03.134091 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.133619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40744)
0810 14:44:03.133816 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.133940 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 14:44:03.135119 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.134755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40754)
0810 14:44:03.134907 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.134980 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 14:44:03.135310 26027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:34575: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
I20260810 14:44:03.203608 26027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } attempt: 2
I20260810 14:44:03.205523 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.204778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40758)
0810 14:44:03.205223 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.205350 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260810 14:44:03.206816 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.206384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40760)
0810 14:44:03.206602 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.206690 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260810 14:44:03.207015 26027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:34575: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
I20260810 14:44:03.298259 26027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } attempt: 3
I20260810 14:44:03.299455 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.299002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40772)
0810 14:44:03.299200 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.299313 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 14:44:03.300588 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.300174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40774)
0810 14:44:03.300356 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.300428 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260810 14:44:03.300786 26027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:34575: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
I20260810 14:44:03.444105 26027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } attempt: 4
I20260810 14:44:03.445338 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.444799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40782)
0810 14:44:03.445118 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.445199 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260810 14:44:03.446345 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.446017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40792)
0810 14:44:03.446166 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.446228 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 14:44:03.446519 26027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.125:34575: Network error: Client connection negotiation failed: client connection to 127.21.55.125:34575: connect: Connection refused (error 111)
W20260810 14:44:03.472118 26040 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:44:03.472472 26040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:03.472607 26040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:03.472741 26040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:03.480130 26040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:03.480250 26040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:03.480372 26040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:03.480450 26040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:03.480546 26040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:03.493724 26040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:36177,127.21.55.125:34575,127.21.55.124:38775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.125.keytab
--principal=kudu/127.21.55.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.125:34575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.55.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:03.495103 26040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:03.496496 26040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:03.510481 26048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:03.510408 26049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:03.510509 26054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:03.511132 26040 server_base.cc:1034] running on GCE node
I20260810 14:44:03.512074 26040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:03.513439 26040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:03.514673 26040 hybrid_clock.cc:648] HybridClock initialized: now 1786373043514607 us; error 48 us; skew 500 ppm
I20260810 14:44:03.515014 26040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:03 dist-test-slave-h6n0 krb5kdc[25895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373043, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:03.518707 26040 init.cc:377] Logged in from keytab as kudu/127.21.55.125@KRBTEST.COM (short username kudu)
I20260810 14:44:03.520181 26040 webserver.cc:460] Webserver started at http://127.21.55.125:35387/ using document root <none> and password file <none>
I20260810 14:44:03.520829 26040 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:03.520920 26040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:03.521191 26040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:03.524178 26040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/data/instance:
uuid: "95eabdf72233440abc6588860106e9bb"
format_stamp: "Formatted at 2026-08-10 14:44:03 on dist-test-slave-h6n0"
server_key: "39cab3e89efa6d0b51ac6c6cee7ee606"
server_key_iv: "75938d139db47494dcb61b1e0c149207"
server_key_version: "encryptionkey@0"
I20260810 14:44:03.524935 26040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/wal/instance:
uuid: "95eabdf72233440abc6588860106e9bb"
format_stamp: "Formatted at 2026-08-10 14:44:03 on dist-test-slave-h6n0"
server_key: "39cab3e89efa6d0b51ac6c6cee7ee606"
server_key_iv: "75938d139db47494dcb61b1e0c149207"
server_key_version: "encryptionkey@0"
I20260810 14:44:03.529955 26040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 14:44:03.533450 26064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:03.535156 26040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:03.535337 26040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/wal
uuid: "95eabdf72233440abc6588860106e9bb"
format_stamp: "Formatted at 2026-08-10 14:44:03 on dist-test-slave-h6n0"
server_key: "39cab3e89efa6d0b51ac6c6cee7ee606"
server_key_iv: "75938d139db47494dcb61b1e0c149207"
server_key_version: "encryptionkey@0"
I20260810 14:44:03.535557 26040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:03.569885 26040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:03.574608 26040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:03.574898 26040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:03.596174 26147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.125:34575 every 8 connection(s)
I20260810 14:44:03.596170 26040 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.125:34575
I20260810 14:44:03.597626 26040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/data/info.pb
I20260810 14:44:03.601361 26148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:03.607926 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 26040
I20260810 14:44:03.608165 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-1/wal/instance
I20260810 14:44:03.608749 21725 external_mini_cluster.cc:1428] Setting key 13e099c2b4d047217b864646c454cc2c
I20260810 14:44:03.607946 26148 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:44:03 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373043, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:44:03.634636 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.611264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55594 (local address 127.21.55.126:36177)
0810 14:44:03.611810 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:03.611828 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:03.612804 (+ 976us) server_negotiation.cc:407] Connection header received
0810 14:44:03.615874 (+ 3070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:03.615878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:03.615946 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:03.616055 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:03.618209 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.619036 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.621464 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.621926 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.622043 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:03.626778 (+ 4735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:03.626787 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:03.626795 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:03.626857 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:03.630696 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:03.631456 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:03.631464 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:03.631470 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:03.631642 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:03.632055 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:03.632062 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:03.632068 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:03.632506 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:03.632690 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:03.633869 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0810 14:44:03.634123 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":274,"threads_started":1}
I20260810 14:44:03.635335 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.611771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:36177 (local address 127.0.0.1:55594)
0810 14:44:03.612619 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.612677 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:03.615540 (+ 2863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:03.616313 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:03.616326 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:03.617165 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:03.617960 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:03.617988 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.619208 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.619216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:03.621278 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:03.621296 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.622177 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.622187 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:03.622503 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:03.623317 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:03.623352 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:03.626363 (+ 3011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:03.630853 (+ 4490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:03.630860 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:03.630872 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:03.631323 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:03.631758 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:03.631761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:03.631763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:03.631939 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:03.632824 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:03.632834 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:03.633428 (+ 594us) client_negotiation.cc:769] Sending connection context
0810 14:44:03.633727 (+ 299us) client_negotiation.cc:240] Negotiation successful
0810 14:44:03.634134 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":345,"threads_started":1}
I20260810 14:44:03.637115 26148 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.55.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.55.124@KRBTEST.COM" created.
Aug 10 14:44:03 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373043, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:44:03.655087 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.637975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40804)
0810 14:44:03.638235 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.638260 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:03.638366 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:03.639509 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:03.639512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:03.639971 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:03.640156 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:03.640165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.642637 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.642650 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:03.643815 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:03.643830 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.645054 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.645059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:03.645211 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:03.645790 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:03.645807 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:03.648789 (+ 2982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:03.652571 (+ 3782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:03.652578 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:03.652582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:03.652946 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:03.653348 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:03.653353 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:03.653357 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:03.653491 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:03.654080 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:03.654085 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:03.654228 (+ 143us) client_negotiation.cc:769] Sending connection context
0810 14:44:03.654387 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 14:44:03.654692 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 14:44:03.655212 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.638174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40804 (local address 127.21.55.125:34575)
0810 14:44:03.638992 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:03.639003 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:03.639035 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:44:03.639094 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:03.639105 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:03.639349 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:03.639509 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:03.640335 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.642256 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.643986 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.644905 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.645008 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:03.649158 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:03.649179 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:03.649206 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:03.649258 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:03.652413 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:03.653092 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:03.653102 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:03.653107 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:03.653234 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:03.653607 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:03.653613 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:03.653617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:03.653952 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:03.654169 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:03.654556 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 14:44:03.654814 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":622,"thread_start_us":471,"threads_started":1}
I20260810 14:44:03.657457 26148 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:44:03.658656 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.658131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38404)
0810 14:44:03.658291 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.658426 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 14:44:03.658900 26073 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
I20260810 14:44:03.660531 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.660179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38410)
0810 14:44:03.660354 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.660427 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 14:44:03.660768 26148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:38775: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.55.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab.
Entry for principal kudu/127.21.55.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab.
I20260810 14:44:03.683225 21725 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
/tmp/dist-test-taskwtLR1L/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.124:38775
--webserver_interface=127.21.55.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab
--principal=kudu/127.21.55.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.55.84:43205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.55.126:36177,127.21.55.125:34575,127.21.55.124:38775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:44:03.710201 26148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } attempt: 1
I20260810 14:44:03.711874 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.711213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38420)
0810 14:44:03.711493 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.711668 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260810 14:44:03.713133 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.712750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38430)
0810 14:44:03.712919 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.713011 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 14:44:03.713492 26148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:38775: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
I20260810 14:44:03.745900 26027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } attempt: 5
Aug 10 14:44:03 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373042, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:44:03.760556 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.747290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40806 (local address 127.21.55.125:34575)
0810 14:44:03.747569 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:03.747576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:03.747603 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:44:03.747670 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:03.747674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:03.747769 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:03.747890 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:03.749129 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.749873 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.751113 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.751579 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.751718 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:03.755068 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:03.755073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:03.755076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:03.755106 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:03.757835 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:03.758486 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:03.758493 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:03.758496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:03.758682 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:03.759067 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:03.759075 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:03.759079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:03.759421 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:03.759580 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:03.759964 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 14:44:03.760188 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 14:44:03.760545 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.746980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40806)
0810 14:44:03.747213 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.747239 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:03.747420 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:03.747937 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:03.747944 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:03.748457 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:03.748844 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:03.748878 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.750027 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.750035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:03.750942 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:03.750954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:03.751747 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:03.751754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:03.751974 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:03.752702 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:03.752720 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:03.754849 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:03.757976 (+ 3127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:03.757982 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:03.757986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:03.758364 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:03.758799 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:03.758804 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:03.758808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:03.758950 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:03.759534 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:03.759539 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:03.759701 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 14:44:03.759851 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 14:44:03.760179 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 14:44:03.762009 26027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:44:03.763233 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.762750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38436)
0810 14:44:03.762942 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.763043 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 14:44:03.764202 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.763881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38452)
0810 14:44:03.764029 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.764110 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 14:44:03.764493 26027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:38775: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
I20260810 14:44:03.781771 26148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } attempt: 2
I20260810 14:44:03.782912 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.782440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38464)
0810 14:44:03.782661 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.782757 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 14:44:03.783885 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.783559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38474)
0810 14:44:03.783699 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.783773 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 14:44:03.784090 26148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:38775: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
I20260810 14:44:03.815846 26027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } attempt: 1
I20260810 14:44:03.817090 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.816615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38476)
0810 14:44:03.816822 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.816924 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 14:44:03.818037 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.817716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38492)
0810 14:44:03.817854 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.817925 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 14:44:03.818282 26027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:38775: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
I20260810 14:44:03.875340 26148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } attempt: 3
I20260810 14:44:03.876775 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.876281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38508)
0810 14:44:03.876529 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.876616 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 14:44:03.877796 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.877406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38512)
0810 14:44:03.877627 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.877697 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260810 14:44:03.878013 26148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:38775: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
I20260810 14:44:03.886489 26027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } attempt: 2
I20260810 14:44:03.887547 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.887103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38524)
0810 14:44:03.887315 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.887390 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 14:44:03.888473 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.888148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38534)
0810 14:44:03.888292 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.888367 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 14:44:03.888671 26027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:38775: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
I20260810 14:44:03.994971 26027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } attempt: 3
I20260810 14:44:03.996567 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.995985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38550)
0810 14:44:03.996259 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.996365 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260810 14:44:03.997819 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:03.997368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38552)
0810 14:44:03.997568 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:03.997640 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260810 14:44:03.998315 26027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:38775: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
I20260810 14:44:04.021242 26148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } attempt: 4
I20260810 14:44:04.022418 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.021932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38560)
0810 14:44:04.022178 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:04.022260 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 14:44:04.023555 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.023200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38576)
0810 14:44:04.023379 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:04.023448 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260810 14:44:04.023809 26148 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.55.124:38775: Network error: Client connection negotiation failed: client connection to 127.21.55.124:38775: connect: Connection refused (error 111)
W20260810 14:44:04.057725 26159 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:44:04.058035 26159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:04.058147 26159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:04.058276 26159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:04.065510 26159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:04.065591 26159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:04.065688 26159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:04.065775 26159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:04.065893 26159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:04.075505 26159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.55.84:43205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.55.126:36177,127.21.55.125:34575,127.21.55.124:38775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/krb5kdc/kudu_127.21.55.124.keytab
--principal=kudu/127.21.55.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.55.124:38775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.55.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:04.076717 26159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:04.078076 26159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:04.092602 26167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:04.092344 26168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:04.092374 26170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:04.094928 26159 server_base.cc:1034] running on GCE node
I20260810 14:44:04.095801 26159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:04.097246 26159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:04.098487 26159 hybrid_clock.cc:648] HybridClock initialized: now 1786373044098424 us; error 36 us; skew 500 ppm
I20260810 14:44:04.098906 26159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:04 dist-test-slave-h6n0 krb5kdc[25895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373044, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:04.102188 26159 init.cc:377] Logged in from keytab as kudu/127.21.55.124@KRBTEST.COM (short username kudu)
I20260810 14:44:04.103698 26159 webserver.cc:460] Webserver started at http://127.21.55.124:37337/ using document root <none> and password file <none>
I20260810 14:44:04.104383 26159 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:04.104517 26159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:04.104847 26159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:04.107923 26159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/data/instance:
uuid: "a872b2a9d8d04d3393840f5019ef5620"
format_stamp: "Formatted at 2026-08-10 14:44:04 on dist-test-slave-h6n0"
server_key: "175aa20e0c6b3cd394386c677e96e911"
server_key_iv: "1f4270ca41d4702e0a34202699d3d516"
server_key_version: "encryptionkey@0"
I20260810 14:44:04.108760 26159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/wal/instance:
uuid: "a872b2a9d8d04d3393840f5019ef5620"
format_stamp: "Formatted at 2026-08-10 14:44:04 on dist-test-slave-h6n0"
server_key: "175aa20e0c6b3cd394386c677e96e911"
server_key_iv: "1f4270ca41d4702e0a34202699d3d516"
server_key_version: "encryptionkey@0"
I20260810 14:44:04.114110 26159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260810 14:44:04.117699 26184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:04.119051 26159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:04.119256 26159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/data,/tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/wal
uuid: "a872b2a9d8d04d3393840f5019ef5620"
format_stamp: "Formatted at 2026-08-10 14:44:04 on dist-test-slave-h6n0"
server_key: "175aa20e0c6b3cd394386c677e96e911"
server_key_iv: "1f4270ca41d4702e0a34202699d3d516"
server_key_version: "encryptionkey@0"
I20260810 14:44:04.119488 26159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:04.138281 26159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:04.144517 26159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:04.144850 26159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:04.166170 26159 rpc_server.cc:307] RPC server started. Bound to: 127.21.55.124:38775
I20260810 14:44:04.166219 26268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.55.124:38775 every 8 connection(s)
I20260810 14:44:04.167953 26159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/data/info.pb
I20260810 14:44:04.172405 26269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:04.174247 21725 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu as pid 26159
I20260810 14:44:04.174463 21725 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwtLR1L/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786372641103011-21725-0/minicluster-data/master-2/wal/instance
I20260810 14:44:04.174950 21725 external_mini_cluster.cc:1428] Setting key 3d708824264116f9be12464d54bcc33b
I20260810 14:44:04.175598 26027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } attempt: 4
I20260810 14:44:04.181666 26269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:44:04 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373042, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.126@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
Aug 10 14:44:04 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
Aug 10 14:44:04 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373044, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:44:04.202584 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.176233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38600)
0810 14:44:04.176509 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:04.176530 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:04.176655 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:04.182361 (+ 5706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:04.182369 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:04.182958 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:04.183195 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:04.183210 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.186121 (+ 2911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.186126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:04.187657 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:04.187673 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.188663 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.188673 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:04.188894 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.189899 (+ 1005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:04.189921 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:04.192855 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:04.198560 (+ 5705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:04.198569 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:04.198575 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:04.199326 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:04.199894 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:04.199901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:04.199905 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:04.200128 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:04.201311 (+ 1183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:04.201317 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:04.201503 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 14:44:04.201760 (+ 257us) client_negotiation.cc:240] Negotiation successful
0810 14:44:04.202106 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260810 14:44:04.204810 26273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.176888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38584 (local address 127.21.55.124:38775)
0810 14:44:04.177896 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.177916 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.179271 (+ 1355us) server_negotiation.cc:407] Connection header received
0810 14:44:04.181464 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.181472 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.181989 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.182402 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.183966 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.185880 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.187806 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.188557 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.188754 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.194206 (+ 5452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:04.194213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:04.194218 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:04.194266 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:04.198802 (+ 4536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.200361 (+ 1559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.200369 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.200376 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.200502 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.201026 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.201031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.201034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.201415 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:04.201723 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.202195 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.203262 (+ 1067us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":630,"server-negotiator.queue_time_us":751,"spinlock_wait_cycles":13312,"thread_start_us":439,"threads_started":1}
I20260810 14:44:04.204892 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.184838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.126:36177 (local address 127.0.0.1:55610)
0810 14:44:04.186205 (+ 1367us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:04.186259 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:04.186433 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:04.186859 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:04.186868 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:04.187472 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:04.187704 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:04.187713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.189151 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.189163 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:04.190442 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:04.190455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.191366 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.191373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:04.191528 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.192273 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:04.192308 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:04.195405 (+ 3097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:04.200453 (+ 5048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:04.200465 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:04.200472 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:04.201074 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:04.201459 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:04.201463 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:04.201467 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:04.201651 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:04.202423 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:04.202430 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:04.202964 (+ 534us) client_negotiation.cc:769] Sending connection context
0810 14:44:04.203350 (+ 386us) client_negotiation.cc:240] Negotiation successful
0810 14:44:04.203678 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":403,"threads_started":1}
I20260810 14:44:04.204941 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.178103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.21.55.124:38775)
0810 14:44:04.178913 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.178918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.178960 (+ 42us) server_negotiation.cc:407] Connection header received
0810 14:44:04.181441 (+ 2481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.181468 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.181991 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.183388 (+ 1397us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.183583 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.185882 (+ 2299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.187861 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.188490 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.189258 (+ 768us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.193144 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:04.193152 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:04.193163 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:04.193192 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:04.198362 (+ 5170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.199572 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.199579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.199586 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.199755 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.200264 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.200272 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.200280 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.201024 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:04.201236 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.202740 (+ 1504us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.203165 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":625,"server-negotiator.queue_time_us":657,"spinlock_wait_cycles":25856,"thread_start_us":473,"threads_started":1}
I20260810 14:44:04.206267 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.184835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55610 (local address 127.21.55.126:36177)
0810 14:44:04.186228 (+ 1393us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.186235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.186353 (+ 118us) server_negotiation.cc:407] Connection header received
0810 14:44:04.186581 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.186590 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.186675 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.186790 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.187902 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.188972 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.190648 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.191109 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.191275 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.195617 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:04.195624 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:04.195629 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:04.195669 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:04.200300 (+ 4631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.201192 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.201200 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.201205 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.201333 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.201799 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.201803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.201806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.202178 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:04.202338 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.205647 (+ 3309us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.205855 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1274,"thread_start_us":1112,"threads_started":1}
I20260810 14:44:04.207815 26269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:44:04.212260 26027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b: Bootstrap starting.
I20260810 14:44:04.216946 26027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:04.218850 26027 log.cc:824] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:44:04 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.126@KRBTEST.COM
I20260810 14:44:04.224368 26027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b: No bootstrap required, opened a new log
Aug 10 14:44:04 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373044, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:44:04.231199 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.208261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55626 (local address 127.21.55.126:36177)
0810 14:44:04.208449 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.208455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.208540 (+ 85us) server_negotiation.cc:407] Connection header received
0810 14:44:04.208689 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.208694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.208795 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.208909 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.210143 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.211328 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.213369 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.214098 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.214256 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.221148 (+ 6892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:04.221157 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:04.221162 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:04.221206 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:04.226836 (+ 5630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.228271 (+ 1435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.228280 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.228285 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.228567 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.229088 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.229093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.229097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.229631 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:04.230030 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.230378 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.230772 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:44:04.232302 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.211123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.125:34575 (local address 127.0.0.1:40818)
0810 14:44:04.211323 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:04.211345 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:04.211473 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:04.215708 (+ 4235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:04.215717 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:04.216349 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:04.216627 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:04.216645 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.218246 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.218256 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:04.219766 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:04.219800 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.221678 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.221802 (+ 124us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:04.222037 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.223095 (+ 1058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:04.223146 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:04.225590 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:04.229370 (+ 3780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:04.229376 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:04.229380 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:04.229844 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:04.230346 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:04.230350 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:04.230353 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:04.230575 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:04.231223 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:04.231228 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:04.231451 (+ 223us) client_negotiation.cc:769] Sending connection context
0810 14:44:04.231593 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 14:44:04.231888 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 14:44:04.233834 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.214941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40818 (local address 127.21.55.125:34575)
0810 14:44:04.215220 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.215227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.215254 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:44:04.215339 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.215345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.215449 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.215588 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.216862 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.218019 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.220102 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.221176 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.221658 (+ 482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.225841 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:04.225847 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:04.225851 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:04.225886 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:04.229220 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.230005 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.230011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.230014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.230207 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.230729 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.230733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.230736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.231094 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:04.231258 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.232927 (+ 1669us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.233363 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 14:44:04.235538 26269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:44:04.238116 26027 raft_consensus.cc:348] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } }
I20260810 14:44:04.238524 26027 raft_consensus.cc:729] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b806ab558629443e818d5b33ba40d13b, State: Initialized, Role: FOLLOWER
I20260810 14:44:04.239078 26027 consensus_queue.cc:260] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } }
I20260810 14:44:04.240059 26282 sys_catalog.cc:455] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } } }
I20260810 14:44:04.240643 26282 sys_catalog.cc:458] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:44:04.242210 26027 sys_catalog.cc:564] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 14:44:04 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373044, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.124@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
Aug 10 14:44:04 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.55.125@KRBTEST.COM
I20260810 14:44:04.255745 26273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.237014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38616 (local address 127.21.55.124:38775)
0810 14:44:04.237147 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.237152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.237289 (+ 137us) server_negotiation.cc:407] Connection header received
0810 14:44:04.237476 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.237481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.237547 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.237715 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.240110 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.241293 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.242661 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.243429 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.243589 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.248656 (+ 5067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:04.248665 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:04.248673 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:04.248725 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:04.252569 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.253379 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.253388 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.253394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.253541 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.253984 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.253989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.253992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.254438 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:04.254612 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.255116 (+ 504us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.255361 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 14:44:04.256634 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.236723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38616)
0810 14:44:04.237200 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:04.237214 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:04.237365 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:04.237732 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:04.237736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:04.238267 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:04.238425 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:04.238434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.241433 (+ 2999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.241436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:04.242380 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:04.242398 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.243634 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.243640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:04.244352 (+ 712us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.244981 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:04.245009 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:04.248418 (+ 3409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:04.252720 (+ 4302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:04.252726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:04.252730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:04.253253 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:04.253694 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:04.253698 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:04.253701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:04.253864 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:04.254703 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:04.254708 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:04.254857 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 14:44:04.255985 (+ 1128us) client_negotiation.cc:240] Negotiation successful
0810 14:44:04.256292 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275}
I20260810 14:44:04.257094 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.234817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.21.55.125:34575)
0810 14:44:04.235014 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.235021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.235054 (+ 33us) server_negotiation.cc:407] Connection header received
0810 14:44:04.235115 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.235120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.235368 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.235512 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.238630 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.239880 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.242461 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.243128 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.243295 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.248918 (+ 5623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:04.248926 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:04.248932 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:04.248980 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:04.252801 (+ 3821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.253712 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.253717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.253720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.253872 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.254362 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.254367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.254370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.254785 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:04.256163 (+ 1378us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.256302 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.256539 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 14:44:04.263556 26298 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:44:04.264002 25948 tablet.cc:2369] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:44:04.264780 26269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620: Bootstrap starting.
W20260810 14:44:04.264858 26298 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:44:04.265038 21725 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:44:04.267452 25980 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b806ab558629443e818d5b33ba40d13b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55626
I20260810 14:44:04.267860 25980 raft_consensus.cc:482] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 14:44:04.268079 25980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:04.268957 26269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:04.270279 26269 log.cc:824] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:04.271764 25980 raft_consensus.cc:504] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } }
I20260810 14:44:04.256208 25921 master_runner.cc:186] Error getting master registration for 127.21.55.125:34575: OK, instance_id { permanent_uuid: "95eabdf72233440abc6588860106e9bb" instance_seqno: 1786373043590430 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:44:04.272441 25921 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:44:04.275113 25980 leader_election.cc:288] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [CANDIDATE]: Term 1 election: Requested vote from peers 95eabdf72233440abc6588860106e9bb (127.21.55.125:34575), a872b2a9d8d04d3393840f5019ef5620 (127.21.55.124:38775)
I20260810 14:44:04.277103 26269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620: No bootstrap required, opened a new log
I20260810 14:44:04.293682 26269 raft_consensus.cc:348] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } }
I20260810 14:44:04.294275 26269 raft_consensus.cc:729] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a872b2a9d8d04d3393840f5019ef5620, State: Initialized, Role: FOLLOWER
I20260810 14:44:04.295078 26269 consensus_queue.cc:260] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } }
I20260810 14:44:04.297708 26304 sys_catalog.cc:455] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } } }
I20260810 14:44:04.298686 26304 sys_catalog.cc:458] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:44:04.299276 26269 sys_catalog.cc:564] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:04.299324 26225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b806ab558629443e818d5b33ba40d13b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a872b2a9d8d04d3393840f5019ef5620"
I20260810 14:44:04.299988 26225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:04.303797 26225 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b806ab558629443e818d5b33ba40d13b in term 1.
I20260810 14:44:04.305296 25946 leader_election.cc:302] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a872b2a9d8d04d3393840f5019ef5620, b806ab558629443e818d5b33ba40d13b; no voters:
I20260810 14:44:04.305855 26282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:04.307138 26282 raft_consensus.cc:686] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [term 1 LEADER]: Becoming Leader. State: Replica: b806ab558629443e818d5b33ba40d13b, State: Running, Role: LEADER
I20260810 14:44:04.307717 26282 consensus_queue.cc:237] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } }
I20260810 14:44:04.323035 26148 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } attempt: 5
Aug 10 14:44:04 dist-test-slave-h6n0 krb5kdc[25895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373043, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.55.125@KRBTEST.COM for kudu/127.21.55.124@KRBTEST.COM
I20260810 14:44:04.341173 26308 sys_catalog.cc:455] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b806ab558629443e818d5b33ba40d13b. Latest consensus state: current_term: 1 leader_uuid: "b806ab558629443e818d5b33ba40d13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } } }
I20260810 14:44:04.341615 26308 sys_catalog.cc:458] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:04.349716 26327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:04.352710 26327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:04.360618 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.323750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.55.124:38775 (local address 127.0.0.1:38630)
0810 14:44:04.324102 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:04.324123 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:04.324256 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:04.325191 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:04.325197 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:04.325801 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:04.326036 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:04.326051 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.327677 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.327683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:04.329138 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:04.329154 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.330453 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.330460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:04.330739 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.331577 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:04.331622 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:04.335712 (+ 4090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:04.357240 (+ 21528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:04.357246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:04.357250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:04.358056 (+ 806us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:04.358575 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:04.358582 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:04.358587 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:04.358758 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:04.359553 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:04.359559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:04.359731 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 14:44:04.359891 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 14:44:04.360194 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235}
I20260810 14:44:04.362165 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.324484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38630 (local address 127.21.55.124:38775)
0810 14:44:04.324667 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.324673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.324693 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:44:04.324756 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.324761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.324840 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.324951 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.326260 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.327477 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.329434 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.330068 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.330189 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.335994 (+ 5805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:04.336002 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:04.336007 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:04.336049 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:04.356762 (+ 20713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.358229 (+ 1467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.358235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.358239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.358389 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.358900 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.358905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.358908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.359320 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:04.359486 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.361562 (+ 2076us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.361843 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 14:44:04.364500 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.317417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40834 (local address 127.21.55.125:34575)
0810 14:44:04.317697 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.317707 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.324197 (+ 6490us) server_negotiation.cc:407] Connection header received
0810 14:44:04.330482 (+ 6285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.330492 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.330637 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.330734 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.336953 (+ 6219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.338054 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.340168 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.340803 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.340893 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.344858 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:04.344866 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:04.344871 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:04.344915 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:04.349231 (+ 4316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.361649 (+ 12418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.361659 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.361665 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.361882 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.362356 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.362363 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.362371 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.362815 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:04.363083 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.363793 (+ 710us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.364086 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 14:44:04.366060 26273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.322371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.21.55.124:38775)
0810 14:44:04.322591 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.322597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.325285 (+ 2688us) server_negotiation.cc:407] Connection header received
0810 14:44:04.326226 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.326233 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.326330 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.326416 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.332552 (+ 6136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.333549 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.337226 (+ 3677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.337909 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.338001 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.350477 (+ 12476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:04.350489 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:04.350495 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:04.350561 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:04.361358 (+ 10797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.362856 (+ 1498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.362864 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.362871 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.363024 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.363716 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.363722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.363726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.364129 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:04.364379 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.365442 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.365715 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:44:04.369372 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.309239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55642 (local address 127.21.55.126:36177)
0810 14:44:04.309415 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.309422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.321586 (+ 12164us) server_negotiation.cc:407] Connection header received
0810 14:44:04.327485 (+ 5899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.327493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.327588 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.327678 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.333388 (+ 5710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.334584 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.339917 (+ 5333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.340679 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.340782 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.346734 (+ 5952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:04.346745 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:04.346753 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:04.346808 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:04.352607 (+ 5799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.355366 (+ 2759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.355376 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.355384 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.355623 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:04.356561 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:04.356569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:04.356573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:04.357045 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:04.363233 (+ 6188us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.368374 (+ 5141us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.368746 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 14:44:04.372684 26148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb: Bootstrap starting.
I20260810 14:44:04.379073 26148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:04.383533 26148 log.cc:824] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:04.391438 26338 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 14:44:04.393389 26225 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [term 1 FOLLOWER]: Refusing update from remote peer b806ab558629443e818d5b33ba40d13b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:04.395040 26308 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 14:44:04.397425 26191 tablet.cc:2369] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:44:04.412807 26304 sys_catalog.cc:455] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b806ab558629443e818d5b33ba40d13b. Latest consensus state: current_term: 1 leader_uuid: "b806ab558629443e818d5b33ba40d13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } } }
I20260810 14:44:04.413089 26304 sys_catalog.cc:458] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:44:04.417008 26148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb: No bootstrap required, opened a new log
I20260810 14:44:04.401950 26159 master_runner.cc:186] Error getting master registration for 127.21.55.125:34575: OK, instance_id { permanent_uuid: "95eabdf72233440abc6588860106e9bb" instance_seqno: 1786373043590430 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:44:04.423663 26159 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:44:04.424731 26308 sys_catalog.cc:455] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b806ab558629443e818d5b33ba40d13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } } }
I20260810 14:44:04.425096 26308 sys_catalog.cc:458] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:04.433528 26327 catalog_manager.cc:1347] Generated new cluster ID: 53448f8b62c94afbb8116f6d4ba134f2
I20260810 14:44:04.433668 26327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:04.435075 26304 sys_catalog.cc:455] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b806ab558629443e818d5b33ba40d13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } } }
I20260810 14:44:04.435316 26304 sys_catalog.cc:458] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:44:04.441200 26148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } }
I20260810 14:44:04.441603 26148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95eabdf72233440abc6588860106e9bb, State: Initialized, Role: FOLLOWER
I20260810 14:44:04.442132 26148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } }
I20260810 14:44:04.443408 26344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } } }
I20260810 14:44:04.444154 26344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:44:04.444844 26148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:04.445185 26111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:04.448833 26112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b806ab558629443e818d5b33ba40d13b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "95eabdf72233440abc6588860106e9bb"
I20260810 14:44:04.450619 26111 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [term 1 FOLLOWER]: Refusing update from remote peer b806ab558629443e818d5b33ba40d13b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:04.455529 26112 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b806ab558629443e818d5b33ba40d13b in term 1.
I20260810 14:44:04.457006 26282 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [LEADER]: Connected to new peer: Peer: permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:04.462738 26327 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:04.465076 26327 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:04.469590 26351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b806ab558629443e818d5b33ba40d13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } } }
I20260810 14:44:04.470031 26351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:44:04.469934 26344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b806ab558629443e818d5b33ba40d13b. Latest consensus state: current_term: 1 leader_uuid: "b806ab558629443e818d5b33ba40d13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } } }
I20260810 14:44:04.470202 26344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:44:04.471246 26308 sys_catalog.cc:455] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b806ab558629443e818d5b33ba40d13b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b806ab558629443e818d5b33ba40d13b" member_type: VOTER last_known_addr { host: "127.21.55.126" port: 36177 } } peers { permanent_uuid: "95eabdf72233440abc6588860106e9bb" member_type: VOTER last_known_addr { host: "127.21.55.125" port: 34575 } } peers { permanent_uuid: "a872b2a9d8d04d3393840f5019ef5620" member_type: VOTER last_known_addr { host: "127.21.55.124" port: 38775 } } }
I20260810 14:44:04.471552 26308 sys_catalog.cc:458] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:04.482990 26327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b806ab558629443e818d5b33ba40d13b: Generated new TSK 0
I20260810 14:44:04.483942 26327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:44:04.506238 26370 catalog_manager.cc:1259] Loaded cluster ID: 53448f8b62c94afbb8116f6d4ba134f2
I20260810 14:44:04.506449 26370 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb: loading cluster ID for follower catalog manager: success
I20260810 14:44:04.510710 26370 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb: acquiring CA information for follower catalog manager: success
I20260810 14:44:04.514240 26370 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 95eabdf72233440abc6588860106e9bb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 14:44:04.595002 26155 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:04.584950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40844 (local address 127.21.55.125:34575)
0810 14:44:04.585333 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.585339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.591789 (+ 6450us) server_negotiation.cc:407] Connection header received
0810 14:44:04.593294 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.593300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.593412 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.593498 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.594767 (+ 1269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40844: BlockingRecv error: recv got EOF from 127.0.0.1:40844 (error 108)
Metrics: {"server-negotiator.queue_time_us":258}
W20260810 14:44:04.597631 26276 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:04.584748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55644 (local address 127.21.55.126:36177)
0810 14:44:04.585178 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.585185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.595293 (+ 10108us) server_negotiation.cc:407] Connection header received
0810 14:44:04.595622 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.595632 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.595751 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.595854 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.597287 (+ 1433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55644: BlockingRecv error: recv got EOF from 127.0.0.1:55644 (error 108)
Metrics: {"server-negotiator.queue_time_us":271}
W20260810 14:44:04.600630 26274 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:04.592426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.21.55.124:38775)
0810 14:44:04.592616 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.592622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.598592 (+ 5970us) server_negotiation.cc:407] Connection header received
0810 14:44:04.598672 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.598677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.598764 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.598864 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.600300 (+ 1436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38632: BlockingRecv error: recv got EOF from 127.0.0.1:38632 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:44:04.639923 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.630864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40850 (local address 127.21.55.125:34575)
0810 14:44:04.631065 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.631073 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.631654 (+ 581us) server_negotiation.cc:407] Connection header received
0810 14:44:04.631881 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.631886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.632101 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.632291 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:04.632777 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.634134 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.636343 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.637158 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.637334 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.637991 (+ 657us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:04.638950 (+ 959us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:04.639162 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.639347 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.639489 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 14:44:04.652087 26274 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:04.649585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38634 (local address 127.21.55.124:38775)
0810 14:44:04.649799 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.649805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.650458 (+ 653us) server_negotiation.cc:407] Connection header received
0810 14:44:04.650686 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.650692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.650857 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.650978 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.651871 (+ 893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38634: BlockingRecv error: recv got EOF from 127.0.0.1:38634 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:44:04.669919 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.660094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.21.55.125:34575)
0810 14:44:04.660270 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.660281 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.662766 (+ 2485us) server_negotiation.cc:407] Connection header received
0810 14:44:04.663031 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.663037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.663167 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.663287 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:04.663890 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.665085 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.667359 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.668179 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.668331 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.668914 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:04.669090 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:04.669214 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.669423 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.669512 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 14:44:04.682013 26274 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:04.679450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38648 (local address 127.21.55.124:38775)
0810 14:44:04.679668 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.679676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.680398 (+ 722us) server_negotiation.cc:407] Connection header received
0810 14:44:04.680649 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.680655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.680750 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.680868 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.681758 (+ 890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38648: BlockingRecv error: recv got EOF from 127.0.0.1:38648 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 14:44:04.694942 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.687838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.21.55.125:34575)
0810 14:44:04.688025 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.688031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.688556 (+ 525us) server_negotiation.cc:407] Connection header received
0810 14:44:04.688743 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.688749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.688836 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.688938 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:04.689366 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.690566 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.692675 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.693371 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.693513 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.694029 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:04.694178 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:04.694268 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.694465 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.694591 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 14:44:04.704370 26274 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:04.701842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38664 (local address 127.21.55.124:38775)
0810 14:44:04.702057 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.702066 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.702626 (+ 560us) server_negotiation.cc:407] Connection header received
0810 14:44:04.702838 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.702844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.702935 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.703035 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.704087 (+ 1052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38664: BlockingRecv error: recv got EOF from 127.0.0.1:38664 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 14:44:04.719558 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.712469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40868 (local address 127.21.55.125:34575)
0810 14:44:04.712725 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.712732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.713268 (+ 536us) server_negotiation.cc:407] Connection header received
0810 14:44:04.713469 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.713475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.713572 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.713668 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:04.714154 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.715348 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.717371 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.718088 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.718217 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.718728 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:04.718867 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:04.718976 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.719166 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.719251 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
W20260810 14:44:04.729985 26274 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:04.727568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38668 (local address 127.21.55.124:38775)
0810 14:44:04.727866 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.727873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.728406 (+ 533us) server_negotiation.cc:407] Connection header received
0810 14:44:04.728616 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.728622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.728716 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.728826 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.729732 (+ 906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38668: BlockingRecv error: recv got EOF from 127.0.0.1:38668 (error 108)
Metrics: {"server-negotiator.queue_time_us":168}
I20260810 14:44:04.750059 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.742391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40874 (local address 127.21.55.125:34575)
0810 14:44:04.742716 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.742722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.743249 (+ 527us) server_negotiation.cc:407] Connection header received
0810 14:44:04.743458 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.743469 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.743563 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.743665 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:04.744184 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.745420 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.747584 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.748438 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.748583 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.749151 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:04.749303 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:04.749397 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.749614 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.749708 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
W20260810 14:44:04.759650 26274 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:04.757132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38684 (local address 127.21.55.124:38775)
0810 14:44:04.757322 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.757330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.758021 (+ 691us) server_negotiation.cc:407] Connection header received
0810 14:44:04.758276 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.758282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.758382 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.758500 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.759424 (+ 924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38684: BlockingRecv error: recv got EOF from 127.0.0.1:38684 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:44:04.792558 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.784623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40876 (local address 127.21.55.125:34575)
0810 14:44:04.784834 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.784978 (+ 144us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.785985 (+ 1007us) server_negotiation.cc:407] Connection header received
0810 14:44:04.786197 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.786204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.786298 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.786400 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:04.786957 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.788243 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.790319 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.791057 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.791184 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.791696 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:04.791888 (+ 192us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:04.791986 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.792150 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.792248 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260810 14:44:04.804749 26274 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:04.800854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38692 (local address 127.21.55.124:38775)
0810 14:44:04.801090 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.801097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.803151 (+ 2054us) server_negotiation.cc:407] Connection header received
0810 14:44:04.803342 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.803349 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.803471 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.803597 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.804499 (+ 902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38692: BlockingRecv error: recv got EOF from 127.0.0.1:38692 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:44:04.848563 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.841736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40878 (local address 127.21.55.125:34575)
0810 14:44:04.841999 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.842008 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.842367 (+ 359us) server_negotiation.cc:407] Connection header received
0810 14:44:04.842584 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.842590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.842706 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.842831 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:04.843256 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.844471 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.846066 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.846923 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.847042 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.847601 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:04.847719 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:04.847868 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.848017 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.848117 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 14:44:04.860025 26274 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:04.856801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38698 (local address 127.21.55.124:38775)
0810 14:44:04.857032 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.857038 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.857725 (+ 687us) server_negotiation.cc:407] Connection header received
0810 14:44:04.858234 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.858241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.858340 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.858517 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.859721 (+ 1204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38698: BlockingRecv error: recv got EOF from 127.0.0.1:38698 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 14:44:04.934775 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:04.928101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40888 (local address 127.21.55.125:34575)
0810 14:44:04.928341 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.928348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.928738 (+ 390us) server_negotiation.cc:407] Connection header received
0810 14:44:04.928957 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.928961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.929041 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.929122 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:04.929614 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.930367 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.932474 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:04.932938 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:04.933057 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:04.933558 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:04.933673 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:04.933741 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:04.934077 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 14:44:04.934204 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260810 14:44:04.944636 26274 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:04.942142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38702 (local address 127.21.55.124:38775)
0810 14:44:04.942377 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:04.942386 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:04.942795 (+ 409us) server_negotiation.cc:407] Connection header received
0810 14:44:04.942922 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:04.942928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:04.943028 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:04.943157 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:04.944306 (+ 1149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38702: BlockingRecv error: recv got EOF from 127.0.0.1:38702 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 14:44:05.083930 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:05.077303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40902 (local address 127.21.55.125:34575)
0810 14:44:05.077602 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:05.077610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:05.077947 (+ 337us) server_negotiation.cc:407] Connection header received
0810 14:44:05.078326 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:05.078331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:05.078482 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:05.078651 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:05.079176 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.080042 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.081996 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.082505 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.082675 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:05.083135 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:05.083252 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:05.083356 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:05.083525 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 14:44:05.083629 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260810 14:44:05.092118 26274 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:05.090046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38704 (local address 127.21.55.124:38775)
0810 14:44:05.090296 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:05.090302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:05.090596 (+ 294us) server_negotiation.cc:407] Connection header received
0810 14:44:05.090830 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:05.090834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:05.090910 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:05.090995 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:05.091920 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38704: BlockingRecv error: recv got EOF from 127.0.0.1:38704 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 14:44:05.356853 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:05.352036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40914 (local address 127.21.55.125:34575)
0810 14:44:05.352277 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:05.352282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:05.352411 (+ 129us) server_negotiation.cc:407] Connection header received
0810 14:44:05.352575 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:05.352579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:05.352668 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:05.352748 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:05.353110 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.353854 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.355268 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.355667 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.355762 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:05.356147 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:05.356253 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:05.356322 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:05.356467 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 14:44:05.356530 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260810 14:44:05.364814 26274 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:05.362962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38720 (local address 127.21.55.124:38775)
0810 14:44:05.363254 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:05.363258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:05.363435 (+ 177us) server_negotiation.cc:407] Connection header received
0810 14:44:05.363569 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:05.363572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:05.363655 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:05.363740 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:05.364537 (+ 797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38720: BlockingRecv error: recv got EOF from 127.0.0.1:38720 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
I20260810 14:44:05.396162 26338 catalog_manager.cc:1259] Loaded cluster ID: 53448f8b62c94afbb8116f6d4ba134f2
I20260810 14:44:05.396440 26338 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620: loading cluster ID for follower catalog manager: success
I20260810 14:44:05.399267 26338 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620: acquiring CA information for follower catalog manager: success
I20260810 14:44:05.401379 26338 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a872b2a9d8d04d3393840f5019ef5620: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:44:05.890504 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:05.883936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40928 (local address 127.21.55.125:34575)
0810 14:44:05.884505 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:05.884511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:05.884649 (+ 138us) server_negotiation.cc:407] Connection header received
0810 14:44:05.884890 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:05.884894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:05.884963 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:05.885088 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:05.885520 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.886264 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.887980 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.888711 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.888869 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:05.889589 (+ 720us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:05.889741 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:05.889860 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:05.890057 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 14:44:05.890193 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":289,"threads_started":1}
I20260810 14:44:05.902851 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:05.896978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.21.55.124:38775)
0810 14:44:05.897434 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:05.897439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:05.897521 (+ 82us) server_negotiation.cc:407] Connection header received
0810 14:44:05.897659 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:05.897662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:05.897830 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:05.897936 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:05.898319 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.899072 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.900514 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.901130 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.901241 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:05.901695 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:05.902242 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:05.902366 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:05.902488 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 14:44:05.902625 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":263,"threads_started":1}
I20260810 14:44:05.916548 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:05.909045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40936 (local address 127.21.55.125:34575)
0810 14:44:05.909274 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:05.909283 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:05.910013 (+ 730us) server_negotiation.cc:407] Connection header received
0810 14:44:05.910318 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:05.910323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:05.910409 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:05.910478 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:05.911369 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.912413 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.914230 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.914734 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.914872 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:05.915331 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:05.915478 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:05.915573 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:05.916146 (+ 573us) server_negotiation.cc:299] Negotiation successful
0810 14:44:05.916236 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 14:44:05.917620 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:05.909182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55646 (local address 127.21.55.126:36177)
0810 14:44:05.910086 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:05.910094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:05.910649 (+ 555us) server_negotiation.cc:407] Connection header received
0810 14:44:05.910950 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:05.910957 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:05.911169 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:05.911357 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:05.912159 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.912979 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.915201 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.915724 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.915839 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:05.916468 (+ 629us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:05.916919 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:05.917045 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:05.917227 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 14:44:05.917312 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":644,"threads_started":1}
I20260810 14:44:05.918462 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:05.910510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38734 (local address 127.21.55.124:38775)
0810 14:44:05.910690 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:05.910695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:05.911418 (+ 723us) server_negotiation.cc:407] Connection header received
0810 14:44:05.911597 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:05.911602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:05.911685 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:05.911771 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:05.912366 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.913424 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.915602 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:05.916236 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:05.916355 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:05.917573 (+ 1218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:05.917687 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:05.917776 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:05.918115 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 14:44:05.918175 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 14:44:05.926949 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 25921
I20260810 14:44:05.928128 26019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:06.041317 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 25921
W20260810 14:44:06.071302 26072 connection.cc:535] server connection from 127.0.0.1:40806 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 14:44:06.072012 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 26040
I20260810 14:44:06.073232 26142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:06.187747 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 26040
I20260810 14:44:06.231817 21725 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu pid 26159
I20260810 14:44:06.233737 26264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:06.371037 21725 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwtLR1L/build/asan/bin/kudu with pid 26159
2026-08-10T14:44:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5630 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10869 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (405245 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST